Arc caching for determininstic finite automata of regular expression accelerator

ABSTRACT

A DFA engine is described that determines whether a current symbol of a payload matches a label of any effective arcs or negative arcs associated with a current node of a DFA graph that are stored in a cache. Responsive to determining that the current symbol does not match a label of any effective or negative arcs associated with the current node of the DFA graph, the DFA engine determines whether the current symbol matches a label of any arc associated with the current node of the DFA graph that is stored in a memory. Responsive to determining that the current symbol matches a label of a particular arc associated with the current node of the DFA graph that is stored in the memory, the DFA engine stores the particular arc in the cache as a new effective arc and uses the particular arc to evaluate the current symbol.

TECHNICAL FIELD

The disclosure relates to processing packets of information, forexample, in the fields of networking and storage.

BACKGROUND

In a typical computer network, a large collection of interconnectedservers provides computing and/or storage capacity for execution ofvarious applications. A data center is one example of a large-scalecomputer network and typically hosts applications and services forsubscribers, i.e., customers of the data center. The data center may,for example, host all of the infrastructure equipment, such as computenodes, networking and storage systems, power systems, and environmentalcontrol systems. In most data centers, clusters of storage systems andapplication servers are interconnected via a high-speed switch fabricprovided by one or more tiers of physical network switches and routers.Data centers vary greatly in size, with some public data centerscontaining hundreds of thousands of servers, and are usually distributedacross multiple geographies for redundancy.

Many devices within a computer network, e.g., storage/compute servers,firewalls, intrusion detection devices, switches, routers or othernetwork attached devices, often use general purpose processors,including multi-core processing systems, to process data, such asnetwork or storage data. However, general purpose processing cores andmulti-processing systems are normally not designed for high-capacitynetwork and storage workloads of modern networks and can be relativelypoor at performing packet stream processing.

SUMMARY

In general, this disclosure describes a highly programmable device,referred to generally as a data processing unit, having multipleprocessing units for processing streams of information, such as networkpackets or storage packets. In some examples, the processing units maybe processing cores, and in other examples, the processing units may bevirtual processors, hardware threads, hardware blocks, or othersub-processing core units. As described herein, the data processing unitincludes one or more specialized hardware-based accelerators configuredto perform acceleration for various data-processing functions, therebyoffloading tasks from the processing units.

In various examples, this disclosure describes a programmable,hardware-based accelerator unit configured to apply and evaluate regularexpressions against high-speed data streams. The accelerator unit mayinclude a hardware implementation of a regular expression (RegEx)evaluator, and thus, may be referred to herein as a RegEx acceleratorunit, or simply a RegEx accelerator. In particular, the RegExaccelerator unit may be configured to process one or more deterministicfinite automata (DFA) to evaluate regular expressions against particulardata units of the data streams. Regular expressions generally define apattern of characters, expressed in a regular language, to be identifiedin an input sequence of characters, such as one or more payloads of oneor more packets. The RegEx accelerator of this disclosure may beconfigured to identify occurrences of one or more target strings definedby one or more respective regular expressions in a set of one or morepayloads of packets using one or more DFAs. The RegEx accelerator may beused as part of various data processing services, such as intrusiondetection and prevention (IDP), anti-virus scanning, search, indexing,and the like.

In one example, a processing device includes a memory, a cache, and aDFA engine. The memory is configured to store at least a portion of adeterministic finite automata (DFA) graph. The cache is configured tostore at least one of: one or more effective arcs or one or morenegative arcs. The DFA engine is implemented in circuitry and includes aprocessing unit. The processing unit is configured to evaluate a payloadby at least: determining whether a current symbol of the payload matchesa label of any of the one or more effective arcs or the one or morenegative arcs associated with a current node of the DFA graph that arestored in the cache; responsive to determining that the current symboldoes not match a label of any one of the one or more effective arcs orany one of the one or more negative arcs associated with the currentnode of the DFA graph, determining whether the current symbol matches alabel of any arc associated with the current node of the DFA graph thatis stored in the memory; and responsive to determining that the currentsymbol matches a label of a particular arc associated with the currentnode of the DFA graph that is stored in the memory, storing theparticular arc in the cache as a new effective arc of the one or moreeffective arcs; and using the particular arc to evaluate the payload.

In another example, a method performed by a processing unit of adeterministic finite automata (DFA) engine of a processing device, forevaluating payloads is described. The method includes determiningwhether a current symbol of a payload matches a label of any of one ormore effective arcs or one or more negative arcs associated with acurrent node of a DFA graph that are stored in a cache of the processingdevice and responsive to determining that the current symbol does notmatch a label of any one of the one or more effective arcs or any one ofthe one or more negative arcs associated with the current node of theDFA graph, determining whether the current symbol matches a label of anyarc associated with the current node of the DFA graph that is stored ina memory of the processing device. The method further includesresponsive to determining that the current symbol matches a label of aparticular arc associated with the current node of the DFA graph that isstored in the memory, storing the particular arc in the cache as a neweffective arc of the one or more effective arcs; and using theparticular arc to evaluate the payload.

In another example, a processing device includes a memory, and a DFAengine. The memory is configured to store: at least a portion of adeterministic finite automata (DFA) graph, the DFA graph comprising aplurality of nodes, each of the nodes having zero or more arcs with eachof the zero or more arcs including a respective label and pointing to arespective subsequent node of the plurality of nodes, at least one ofthe plurality of nodes comprising a match node; one or more arcs of areference node representation for the at least the portion of the DFAgraph; and zero or more arcs of a respective delta node representationfor each node of the at least the portion of the DFA graph other thanthe reference node, the zero or more arcs of each respective delta noderepresentation defining arcs that are not defined by the reference noderepresentation. The DFA engine is implemented in circuitry and includesa processing unit. The processing unit is configured to evaluate apayload by at least: accessing the one or more arcs of reference noderepresentation and the zero or more arcs of the respective delta noderepresentation that is associated with a current node to determinewhether a symbol of the payload matches at least one of: a label of anyof the one or more arcs of the reference node representation; and alabel of any of the zero or more arcs of the respective delta noderepresentation that is associated with the current node; and responsiveto determining that the symbol of the payload matches a label of aparticular arc of the one or more arcs of the reference noderepresentation or the zero or more arcs of the respective delta noderepresentation that is associated with the current node, using theparticular arc to evaluate the payload.

The details of one or more examples are set forth in the accompanyingdrawings and the description below. Other features, objects, andadvantages will be apparent from the description and drawings, and fromthe claims.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating an example system including oneor more network devices configured to efficiently process a series ofwork units in a multiple core processor system.

FIG. 2 is a block diagram illustrating an example data processing unit(DPU) including two or more processing clusters, in accordance with thetechniques of this disclosure.

FIG. 3 is a block diagram illustrating another example data processingunit including two or more processing clusters, in accordance with thetechniques of this disclosure.

FIG. 4 is a block diagram illustrating an example processing clusterincluding a plurality of programmable processing cores.

FIG. 5 is a block diagram illustrating an example regular expression(RegEx) accelerator, in accordance with the techniques of thisdisclosure.

FIG. 6A is a conceptual diagram illustrating an example deterministicfinite automata (DFA) graph.

FIG. 6B is a flowchart illustrating an example method for allocatingdata for nodes of a DFA graph between buffer memory and external memoryof a RegEx accelerator, in accordance with the techniques of thisdisclosure.

FIG. 6C is a conceptual diagram illustrating an example set of memoryslices of a buffer memory, including data for arcs of nodes of a DFAgraph.

FIGS. 7A through 7C are flowcharts illustrating example regularexpression operations performed in accordance with the techniques ofthis disclosure.

FIG. 8 is a flowchart illustrating an example method for traversing aDFA graph from one node to a next node in accordance with the techniquesof this disclosure.

FIG. 9 is a block diagram illustrating an example DFA engine.

FIGS. 10A and 10B are conceptual diagrams illustrating noderepresentations of a DFA graph, in accordance with the techniques ofthis disclosure.

FIG. 10C is a conceptual diagram illustrating an example multi-portcache of an example RegEx accelerator, in accordance with the techniquesof this disclosure.

FIGS. 10D and 10E are conceptual diagrams illustrating logical views ofa cache of an example RegEx accelerator, in accordance with thetechniques of this disclosure.

DETAILED DESCRIPTION

FIG. 1 is a block diagram illustrating an example system 8 including oneor more network devices configured to efficiently process a series ofwork units in a multiple core processor system. As described herein,techniques for caching and prefetching data from non-coherent memory mayprovide technical benefits that include improving the efficiency andutilization of processing cores within access nodes 17 in FIG. 1. Accessnodes may also be referred to as data processing units (DPUs), ordevices including DPUs, in this disclosure. In the example of FIG. 1,various data structures and processing techniques are described withrespect to access nodes 17 within a data center 10. Other devices withina network, such as routers, switches, servers, firewalls, gateways andthe like, having multiple core processor systems may readily beconfigured to utilize the data processing techniques described herein.

Data center 10 represents an example of a system in which varioustechniques described herein may be implemented. In general, data center10 provides an operating environment for applications and services forcustomers 11 coupled to the data center by service provider network 7and gateway device 20. Data center 10 may, for example, hostinfrastructure equipment, such as compute nodes, networking and storagesystems, redundant power supplies, and environmental controls. Serviceprovider network 7 may be coupled to one or more networks administeredby other providers, and may thus form part of a large-scale publicnetwork infrastructure, e.g., the Internet. In other examples, serviceprovider network 7 may be a data center wide-area network (DC WAN),private network or other type of network.

In some examples, data center 10 may represent one of manygeographically distributed network data centers. In the example of FIG.1, data center 10 is a facility that provides information services forcustomers 11. Customers 11 may be collective entities such asenterprises and governments or individuals. For example, a network datacenter may host web services for several enterprises and end users.Other exemplary services may include data storage, virtual privatenetworks, file storage services, data mining services, scientific- orsuper-computing services, and so on.

In the illustrated example, data center 10 includes a set of storagesystems and application servers 12 interconnected via a high-speedswitch fabric 14. In some examples, servers 12 are arranged intomultiple different server groups, each including any number of serversup to, for example, n servers 12 ₁-12 _(n). Servers 12 providecomputation and storage facilities for applications and data associatedwith customers 11 and may be physical (bare-metal) servers, virtualmachines running on physical servers, virtualized containers running onphysical servers, or combinations thereof.

In the example of FIG. 1, each of servers 12 is coupled to switch fabric14 by an access node 17 for processing streams of information, such asnetwork packets or storage packets. In example implementations, accessnodes 17 may be configurable to operate in a standalone networkappliance having one or more access nodes. For example, access nodes 17may be arranged into multiple different access node groups 19, eachincluding any number of access nodes up to, for example, x access nodes17 ₁-17 _(x). In other examples, each access node may be implemented asa component (e.g., electronic chip) within a device, such as a computenode, application server, storage server, and may be deployed on amotherboard of the device or within a removable card, such as a storageand/or network interface card.

In general, each access node group 19 may be configured to operate as ahigh-performance I/O hub designed to aggregate and process networkand/or storage I/O for multiple servers 12. As described above, the setof access nodes 17 within each of the access node groups 19 providehighly-programmable, specialized I/O processing circuits for handlingnetworking and communications operations on behalf of servers 12. Inaddition, in some examples, each of access node groups 19 may includestorage devices 27, such as solid state drives (SSDs) and/or hard diskdrives (HDDs), configured to provide network accessible storage for useby applications executing on the servers 12. In some examples, one ormore of the SSDs may comprise non-volatile memory (NVM) or flash memory.Each access node group 19, including its set of access nodes 17 andstorage devices 27, and the set of servers 12 supported by the accessnodes 17 of that access node group 19 may be referred to herein as anetwork storage compute unit.

As further described herein, in one example, each access node 17 is ahighly programmable I/O processor (referred to as a data processingunit, or DPU) specially designed for offloading certain functions fromservers 12. In one example, each access node 17 includes a number ofinternal processor clusters, each including two or more processing coresand equipped with hardware engines that offload cryptographic functions,compression and decompression, regular expression (RegEx) processing,data storage functions, and networking operations. In this way, eachaccess node 17 includes components for fully implementing and processingnetwork and storage stacks on behalf of one or more servers 12. Inaddition, access nodes 17 may be programmatically configured to serve asa security gateway for its respective servers 12, freeing up theprocessors of the servers to dedicate resources to applicationworkloads. In some example implementations, each access node 17 may beviewed as a network interface subsystem that implements full offload ofthe handling of data packets (with zero copy in server memory) andstorage acceleration for the attached server systems. In one example,each access node 17 may be implemented as one or moreapplication-specific integrated circuit (ASIC) or other hardware andsoftware components, each supporting a subset of the servers. Additionalexample details of various example DPUs are described in U.S.Provisional Patent Application No. 62/559,021, filed Sep. 15, 2017,entitled “Access Node for Data Centers,” and U.S. Provisional PatentApplication No. 62/530,691, filed Jul. 10, 2017, entitled “DataProcessing Unit for Computing Devices,” the entire contents of bothbeing incorporated herein by reference.

In accordance with the techniques of this disclosure, any or all ofaccess nodes 17 may include a regular expression (RegEx) acceleratorunit. That is, one or more computing devices may include an access nodeincluding one or more RegEx accelerator units, according to thetechniques of this disclosure.

The RegEx accelerator unit of the access node, according to thetechniques of this disclosure, may be configured to process payloads ofpackets during various services as the packets are exchanged by accessnodes 17, e.g., between access nodes 17 via switch fabric 14 and/orbetween servers 12. That is, as packets are exchanged between thedevices, either for networking or data storage and retrieval, the accessnode may perform an evaluation service on payloads of the packet. Forexample, the access node may provide evaluation services in the form ofintrusion detection, intrusion prevention, intrusion detection andprevention (IDP), anti-virus scanning, search, indexing, or the like.The access node may use one or more RegEx accelerator units to identifypatterns in payload data, such as virus definitions, attemptedintrusions, search strings, indexing strings, or the like. The patternsmay be defined according to respective regular expressions. According tothe techniques of this disclosure, each of the RegEx accelerator unitsmay include a hardware implementation of a regular expression evaluator,which may construct one or more deterministic finite automata (DFAs)according to the regular expressions for the patterns.

In the example of FIG. 1, each access node 17 provides connectivity toswitch fabric 14 for a different group of servers 12 and may be assignedrespective IP addresses and provide routing operations for the servers12 coupled thereto. Access nodes 17 may interface with and utilizeswitch fabric 14 so as to provide full mesh (any-to-any)interconnectivity such that any of servers 12 may communicate packetdata for a given packet flow to any other of the servers using any of anumber of parallel data paths within the data center 10. In addition,access nodes 17 described herein may provide additional services, suchas storage (e.g., integration of solid-state storage devices), security(e.g., encryption), acceleration (e.g., compression), I/O offloading,and the like. In some examples, one or more of access nodes 17 mayinclude storage devices, such as high-speed solid-state drives orrotating hard drives, configured to provide network accessible storagefor use by applications executing on the servers. More details on theexample data center network architecture and interconnected access nodesillustrated in FIG. 1 are available in U.S. patent application Ser. No.15/939,227, filed Mar. 28, 2018, entitled “Non-Blocking Any-to-Any DataCenter Network with Packet Spraying Over Multiple Alternate Data Paths,”(Attorney Docket No. 1242-002US01), the entire content of which isincorporated herein by reference.

Various example architectures of access nodes 17 are described belowwith respect to FIGS. 2, 3, 4A, and 4B. With respect to either example,the architecture of each access node 17 comprises a multiple coreprocessor system that represents a high performance, hyper-convergednetwork, storage, and data processor and input/output hub. Thearchitecture of each access node 17 is optimized for high performanceand high efficiency stream processing.

In general, a stream, also referred to as a data stream, may be viewedas an ordered, unidirectional sequence of computational objects that canbe of unbounded or undetermined length. In a simple example, a streamoriginates in a producer and terminates at a consumer, is operated onsequentially, and is flow-controlled. In some examples, a stream can bedefined as a sequence of stream fragments, each representing a portionof data communicated by a stream. In one example, a stream fragment mayinclude a memory block contiguously addressable in physical addressspace, an offset into that block, and a valid length. Streams can bediscrete, such as a sequence of packets received from a network, orcontinuous, such as a stream of blocks, words, or bytes read from astorage device. A stream of one type may be transformed into anothertype as a result of processing. Independent of the stream type, streammanipulation requires efficient fragment manipulation. An applicationexecuting on one of access nodes 17 may operate on a stream in threebroad ways: the first is protocol processing, which consists ofoperating on control information or headers within the stream; thesecond is payload processing, which involves significant accessing ofthe data within the stream; and third is some combination of bothcontrol and data access.

Stream processing is a specialized type of conventional general-purposeprocessing supporting specialized limitations with regard to both accessand directionality. Processing typically only accesses a limited portionof the stream at any time, called a “window,” within which it may accessrandom addresses. Objects outside of the window are not accessiblethrough a streaming interface. In contrast, general purpose processingviews the whole memory as randomly accessible at any time. In addition,stream processing generally progresses in one direction, called theforward direction. These characteristics make stream processing amenableto pipelining, as different processors within one of access nodes 17 cansafely access different windows within the stream.

As described herein, data processing units of access nodes 17 mayprocess stream information by managing “work units.” In general, a WorkUnit (WU) is a container that is associated with a stream state and usedto describe (i.e. point to) data within a stream (stored in memory)along with any associated meta-data and operations to be performed onthe data. In the example of FIG. 1, streams of data units maydynamically originate within a peripheral unit of one of access nodes 17(e.g. injected by a networking unit, a host unit, or a solid state driveinterface), or within a processor of the one of access nodes 17, inassociation with one or more streams of data, and terminate at anotherperipheral unit or another processor of the one of access nodes 17. Eachwork unit maintained by a data processing unit is associated with anamount of work that is relevant to the entity executing the work unitfor processing a respective portion of a stream.

Stream processing is typically initiated as a result of receiving one ormore data units associated with respective portions of the stream andconstructing and managing work units for processing respective portionsof the data stream. In protocol processing, a portion would be a singlebuffer (e.g. packet), for example. Within access nodes 17, work unitsmay be executed by processor cores, hardware blocks, I/O interfaces, orother computational processing units. For instance, a processor core ofan access node 17 executes a work unit by accessing the respectiveportion of the stream from memory and performing one or morecomputations in accordance with the work unit. A component of the one ofaccess nodes 17 may receive, execute or generate work units. Asuccession of work units may define how the access node processes aflow, and smaller flows may be stitched together to form larger flows.

For purposes of example, DPUs of or within each access node 17 mayexecute an operating system, such as a general-purpose operating system(e.g., Linux or other flavor of Unix) or a special-purpose operatingsystem, that provides an execution environment for data plane softwarefor data processing. Moreover, each DPU may be configured to utilize awork unit (WU) stack data structure (referred to as a ‘WU stack’ in amultiple core processor system. As described herein, the WU stack datastructure may provide certain technical benefits, such as helping managean event driven, run-to-completion programming model of an operatingsystem executed by the multiple core processor system. The WU stack, ina basic form, may be viewed as a stack of continuation WUs used inaddition to (not instead of) a program stack maintained by the operatingsystem as an efficient means of enabling program execution todynamically move between cores of the access node while performinghigh-rate stream processing. As described below, a WU data structure isa building block in the WU stack and can readily be used to compose aprocessing pipeline and services execution in a multiple core processorsystem. The WU stack structure carries state, memory, and otherinformation in auxiliary variables external to the program stack for anygiven processor core. In some implementations, the WU stack may alsoprovide an exception model for handling abnormal events and a ‘successbypass’ to shortcut a long series of operations. Further, the WU stackmay be used as an arbitrary flow execution model for any combination ofpipelined or parallel processing.

As described herein, access nodes 17 may process WUs through a pluralityof processor cores arranged as processing pipelines within access nodes17, and such processing cores may employ techniques to encourageefficient processing of such work units and high utilization ofprocessing resources. For instance, a processing core (or a processingunit within a core) may, in connection with processing a series of workunits, access data and cache the data into a plurality of segments of alevel 1 cache associated with the processing core. In some examples, aprocessing core may process a work unit and cache data from non-coherentmemory in a segment of the level 1 cache. The processing core may alsoconcurrently prefetch data associated with a work unit expected to beprocessed in the future into another segment of the level 1 cacheassociated with the processing core. By prefetching the data associatedwith the future work unit in advance of the work unit being dequeuedfrom a work unit queue for execution by the core, the processing coremay be able to efficiently and quickly process a work unit once the workunit is dequeued and execution of the work unit is to commence by theprocessing core. More details on work units and stream processing bydata processing units of access nodes are available in U.S. ProvisionalPatent Application No. 62/589,427, filed Nov. 21, 2017, entitled “WorkUnit Stack Data Structures in Multiple Core Processor System,” and U.S.Provisional Patent Application No. 62/625,518, entitled “EFFICIENT WORKUNIT PROCESSING IN A MULTICORE SYSTEM”, filed Feb. 2, 2018, the entirecontents of both being incorporated herein by reference.

As described herein, the data processing units for access nodes 17includes one or more specialized hardware-based accelerators configuredto perform acceleration for various data-processing functions, therebyoffloading tasks from the processing units when processing work units.That is, each accelerator is programmable by the processing cores, andone or more accelerators may be logically chained together to operate onstream data units, such as by providing cryptographic functions,compression and regular expression (RegEx) processing, data storagefunctions and networking operations. This disclosure describes aprogrammable, hardware-based accelerator unit configured to apply andevaluate regular expressions against high-speed data streams. Theaccelerator unit may include a hardware implementation of a regularexpression (RegEx) evaluator, and thus, may be referred to herein as aRegEx accelerator unit, or simply a RegEx accelerator. In particular,the RegEx accelerator unit may be configured to process one or moredeterministic finite automata (DFA) to evaluate regular expressionsagainst particular data units of the data streams.

In some examples, the RegEx accelerator unit is configured tospeculatively access an internal cache of the RegEx accelerator unit toevaluate a payload. That is, the RegEx accelerator unit may include acache (e.g., a L1 cache) that is configured to enable access to multipleportions of the cache simultaneously. For example, the cache may includetwo input ports and two output ports and be configured to concurrentlyreceive two different hash keys and in response, output two differentresults based on the inputted keys in parallel. The RegEx acceleratorunit may store two or more “node representations” of nodes of a DFAgraph in the cache. In this way, rather than access a memory of theRegEx accelerator unit that stores an entire DFA graph, the RegExaccelerator unit may operate more efficiently and with lower latency byaccessing multiple node representations in its internal cache, inparallel.

FIG. 2 is a block diagram illustrating an example data processing unit(DPU) 130 including two or more processing cores, in accordance with thetechniques of this disclosure. DPU 130 generally represents a hardwarechip implemented in digital logic circuitry and may be used in anycomputing or network device. DPU 130 may operate substantially similarto and generally represent any of access nodes 17 of FIG. 1. Thus, DPU130 may be communicatively coupled to one or more network devices,server devices (e.g., servers 12), random access memory, storage media(e.g., solid state drives (SSDs)), a data center fabric (e.g., switchfabric 14), or the like, e.g., via PCI-e, Ethernet (wired or wireless),or other such communication media. Moreover, DPU 130 may be implementedas one or more application-specific integrated circuit (ASIC), may beconfigurable to operate as a component of a network appliance or may beintegrated with other DPUs within a device.

In the illustrated example of FIG. 2, DPU 130 includes a multi-coreprocessor 132 having a plurality of programmable processing cores140A-140N (“cores 140”) coupled to an on-chip memory unit 134. Each ofcores 140 includes a level 1 cache 141 (level 1 caches 141 a, 141 b, and141 n are associated with cores 140 a, 140 b, and 140 n, respectively).

Memory unit 134 may include two types of memory or memory devices,namely coherent cache memory 136 and non-coherent buffer memory 138.Processor 132 also includes a networking unit 142, work unit (WU) queues143, a memory controller 144, and accelerators 146. As illustrated inFIG. 2, each of cores 140, networking unit 142, WU queues 143, memorycontroller 144, memory unit 134, and accelerators 146 arecommunicatively coupled to each other. Processor 132 of DPU 130 furtherincludes one or more accelerators 146 configured to perform accelerationfor various data-processing functions, such as look-ups, matrixmultiplication, cryptography, compression, regular expressions, or thelike.

In this example, DPU 130 represents a high performance, hyper-convergednetwork, storage, and data processor and input/output hub. For example,networking unit 142 may be configured to receive one or more datapackets from and transmit one or more data packets to one or moreexternal devices, e.g., network devices. Networking unit 142 may performnetwork interface card functionality, packet switching, and the like,and may use large forwarding tables and offer programmability.Networking unit 142 may expose Ethernet ports for connectivity to anetwork, such as switch fabric 14 of FIG. 1. DPU 130 may also includeone or more interfaces for connectivity to host devices (e.g., servers)and data storage devices, e.g., solid state drives (SSDs) via PCIelanes. DPU 130 may further include one or more high bandwidth interfacesfor connectivity to off-chip external memory.

Processor 132 further includes accelerators 146 configured to performacceleration for various data-processing functions, such as look-ups,matrix multiplication, cryptography, compression, regular expressions,or the like. For example, accelerators 146 may comprise hardwareimplementations of look-up engines, matrix multipliers, cryptographicengines, compression engines, or the like. The functionality ofdifferent hardware accelerators is described is more detail below withrespect to FIG. 4. In accordance with the techniques of this disclosure,at least one of accelerators 146 represents a hardware implementation ofa regular expression engine. In particular, according to the techniquesof this disclosure, accelerators 146 include at least one RegExaccelerator that includes one or more DFA engines configured to executeDFAs representing regular expressions, as discussed in greater detailbelow.

Memory controller 144 may control access to on-chip memory unit 134 bycores 140, networking unit 142, and any number of external devices,e.g., network devices, servers, external storage devices, or the like.Memory controller 144 may be configured to perform a number ofoperations to perform memory management in accordance with the presentdisclosure. For example, memory controller 144 may be capable of mappingaccesses from one of the cores 140 to either of coherent cache memory136 or non-coherent buffer memory 138. More details on the bifurcatedmemory system included in the DPU are available in U.S. ProvisionalPatent Application No. 62/483,844, filed Apr. 10, 2017, and titled“Relay Consistent Memory Management in a Multiple Processor System,”(Attorney Docket No. FUNG-00200/1242-008USP1), the entire content ofwhich is incorporated herein by reference.

Cores 140 may comprise one or more microprocessors without interlockedpipeline stages (MIPS) cores, reduced instruction set computing (RISC)cores, advanced RISC machine (ARM) cores, performance optimization withenhanced RISC—performance computing (PowerPC) cores, RISC Five (RISC-V)cores, or complex instruction set computing (CISC or x86) cores. Each ofcores 140 may be programmed to process one or more events or activitiesrelated to a given data packet such as, for example, a networking packetor a storage packet. Each of cores 140 may be programmable using ahigh-level programming language, e.g., C, C++, or the like.

Each of level 1 caches 141 may include a plurality of cache lineslogically or physically divided into cache segments. Each of level 1caches 141 may be controlled by a load/store unit also included withinthe core. The load/store unit may include logic for loading data intocache segments and/or cache lines from non-coherent buffer memory 138and/or memory external to DPU 130. The load/store unit may also includelogic for flushing cache segments and/or cache lines to non-coherentbuffer memory 138 and/or memory external to DPU 130. In some examples,the load/store unit may be configured to prefetch data from main memoryduring or after a cache segment or cache line is flushed.

As described herein, processor cores 140 may be arranged as processingpipelines, and such processing cores may employ techniques to encourageefficient processing of such work units and high utilization ofprocessing resources. For instance, any of processing cores 140 (or aprocessing unit within a core) may, in connection with processing aseries of work units retrieved from WU queues 143, access data and cachethe data into a plurality of segments of level 1 cache 141 associatedwith the processing core. In some examples, a processing core 140 mayprocess a work unit and cache data from non-coherent memory 138 in asegment of the level 1 cache 141. As described herein, concurrent withexecution of work units by cores 140, a load store unit of memorycontroller 144 may be configured to prefetch, from non-coherent memory138, data associated with work units within WU queues 143 that areexpected to be processed in the future, e.g., the WUs now at the top ofthe WU queues and next in line to be processed. For each core 140, theload store unit of memory controller 144 may store the prefetched dataassociated with the WU to be processed by the core into a standbysegment of the level 1 cache 141 associated with the processing core140.

In some examples, the plurality of cores 140 executes instructions forprocessing a plurality of events related to each data packet of one ormore data packets, received by networking unit 142, in a sequentialmanner in accordance with one or more work units associated with thedata packets. As described above, work units are sets of data exchangedbetween cores 140 and networking unit 142 where each work unit mayrepresent one or more of the events related to a given data packet.

As one example use case, stream processing may be divided into workunits executed at a number of intermediate processors between source anddestination. Depending on the amount of work to be performed at eachstage, the number and type of intermediate processors that are involvedmay vary. In processing a plurality of events related to each datapacket, a first one of the plurality of cores 140, e.g., core 140A mayprocess a first event of the plurality of events. Moreover, first core140A may provide to a second one of plurality of cores 140, e.g., core140B a first work unit of the one or more work units. Furthermore,second core 140B may process a second event of the plurality of eventsin response to receiving the first work unit from first core 140B.

As another example use case, transfer of ownership of a memory bufferbetween processing cores may be mediated by a work unit messagedelivered to one or more of processing cores 140. For example, the workunit message may be a four-word message including a pointer to a memorybuffer. The first word may be a header containing information necessaryfor message delivery and information used for work unit execution, suchas a pointer to a function for execution by a specified one ofprocessing cores 140. Other words in the work unit message may containparameters to be passed to the function call, such as pointers to datain memory, parameter values, or other information used in executing thework unit.

In one example, receiving a work unit is signaled by receiving a messagein a work unit receive queue (e.g., one of WU queues 143). The one of WUqueues 143 is associated with a processing element, such as one of cores140, and is addressable in the header of the work unit message. One ofcores 140 may generate a work unit message by executing storedinstructions to addresses mapped to a work unit transmit queue (e.g.,another one of WU queues 143). The stored instructions write thecontents of the message to the queue. The release of a work unit messagemay be interlocked with (gated by) flushing of the core's dirty cachedata and in some examples, prefetching into the cache of data associatedwith another work unit for future processing.

FIG. 3 is a block diagram illustrating one example of a DPU 150including a networking unit, at least one host unit, and two or moreprocessing clusters. DPU 150 may operate substantially similar to any ofthe access nodes 17 of FIG. 1. Thus, DPU 150 may be communicativelycoupled to a data center fabric (e.g., switch fabric 14), one or moreserver devices (e.g., servers 12), storage media (e.g., SSDs), one ormore network devices, random access memory, or the like, e.g., viaPCI-e, Ethernet (wired or wireless), or other such communication mediain order to interconnect each of these various elements. DPU 150generally represents a hardware chip implemented in digital logiccircuitry. As various examples, DPU 150 may be provided as an integratedcircuit mounted on a motherboard of a computing, networking and/orstorage device or installed on a card connected to the motherboard ofthe device.

In general, DPU 150 represents a high performance, hyper-convergednetwork, storage, and data processor and input/output hub. Asillustrated in FIG. 3, DPU 150 includes networking unit 152, processingclusters 156A-1 to 156N-M (processing clusters 156), host units 154A-1to 154B-M (host units 154), and central cluster 158, and is coupled toexternal memory 170. Each of host units 154, processing clusters 156,central cluster 158, and networking unit 152 may include a plurality ofprocessing cores, e.g., MIPS cores, ARM cores, PowerPC cores, RISC-Vcores, or CISC or x86 cores. External memory 170 may comprise randomaccess memory (RAM) or dynamic random access memory (DRAM).

As shown in FIG. 3, host units 154, processing clusters 156, centralcluster 158, networking unit 152, and external memory 170 arecommunicatively interconnected via one or more specializednetwork-on-chip fabrics. A set of direct links 162 (represented asdashed lines in FIG. 3) forms a signaling network fabric that directlyconnects central cluster 158 to each of the other components of DPU 150,that is, host units 154, processing clusters 156, networking unit 152,and external memory 170. A set of grid links 160 (represented as solidlines in FIG. 3) forms a data network fabric that connects neighboringcomponents (including host units 154, processing clusters 156,networking unit 152, and external memory 170) to each other in atwo-dimensional grid.

Networking unit 152 has Ethernet interfaces 164 to connect to the switchfabric, and interfaces to the data network formed by grid links 160 andthe signaling network formed by direct links 162. Networking unit 152provides a Layer 3 (i.e., OSI networking model Layer 3) switchforwarding path, as well as network interface card (NIC) assistance. Oneor more hardware direct memory access (DMA) engine instances (not shown)may be attached to the data network ports of networking unit 152, whichare coupled to respective grid links 160. The DMA engines of networkingunit 152 are configured to fetch packet data for transmission. Thepacket data may be in on-chip or off-chip buffer memory (e.g., withinbuffer memory of one of processing clusters 156 or external memory 170),or in host memory.

Host units 154 each have PCI-e interfaces 166 to connect to serversand/or storage devices, such as SSD devices. This allows DPU 150 tooperate as an endpoint or as a root. For example, DPU 150 may connect toa host system (e.g., a server) as an endpoint device, and DPU 150 mayconnect as a root to endpoint devices (e.g., SSD devices). Each of hostunits 154 may also include a respective hardware DMA engine (not shown).Each DMA engine is configured to fetch data and buffer descriptors fromhost memory, and to deliver data and completions to host memory.

DPU 150 provides optimizations for stream processing. DPU 150 executesan operating system that facilitates run-to-completion processing, whichmay eliminate interrupts, thread scheduling, cache thrashing, andassociated costs. For example, an operating system may run on one ormore of processing clusters 156. Central cluster 158 may be configureddifferently from processing clusters 156, which may be referred to asstream processing clusters. In one example, central cluster 158 executesthe operating system kernel (e.g., Linux kernel) as a control plane.Processing clusters 156 may function in run-to-completion thread mode ofa data plane software stack of the operating system. That is, processingclusters 156 may operate in a tight loop fed by work unit queuesassociated with each processing core in a cooperative multi-taskingfashion.

DPU 150 operates on work units (WUs) that associate a buffer with aninstruction stream to reduce dispatching overhead and allow processingby reference to minimize data movement and copy. The stream-processingmodel may structure access by multiple processors (e.g., processingclusters 156) to the same data and resources, avoid simultaneoussharing, and therefore, reduce contention. A processor may relinquishcontrol of data referenced by a work unit as the work unit is passed tothe next processor in line. Central cluster 158 may include a centraldispatch unit responsible for work unit queuing and flow control, workunit and completion notification dispatch, and load balancing andprocessor selection from among processing cores of processing clusters156 and/or central cluster 158.

As described above, work units are sets of data exchanged betweenprocessing clusters 156, networking unit 152, host units 154, centralcluster 158, and external memory 170. Each work unit may be representedby a fixed length data structure, or message, including an action valueand one or more arguments. In one example, a work unit message includesfour words, a first word having a value representing an action value andthree additional words each representing an argument. The action valuemay be considered a work unit message header containing informationnecessary for message delivery and information used for work unitexecution, such as a work unit handler identifier, and source anddestination identifiers of the work unit. The other arguments of thework unit data structure may include a frame argument having a valueacting as a pointer to a continuation work unit to invoke a subsequentwork unit handler, a flow argument having a value acting as a pointer tostate that is relevant to the work unit handler, and a packet argumenthaving a value acting as a packet pointer for packet and/or blockprocessing handlers.

In some examples, one or more processing cores of processing clusters180 may be configured to execute program instructions using a work unit(WU) stack. In general, a work unit (WU) stack is a data structure tohelp manage event driven, run-to-completion programming model of anoperating system typically executed by processing clusters 156 of DPU150, as further described in U.S. Patent Application Ser. No.62/589,427, filed Nov. 21, 2017 (Attorney Docket No. 1242-009USP1), theentire content of which is incorporated herein by reference.

As described herein, in some example implementations, load store unitswithin processing clusters 156 may, concurrent with execution of workunits by cores within the processing clusters, identify work units thatare enqueued in WU queues for future processing by the cores. In someexamples, WU queues storing work units enqueued for processing by thecores within processing clusters 156 may be maintained as hardwarequeues centrally managed by central cluster 158. In such examples, loadstore units may interact with central cluster 158 to identify futurework units to be executed by the cores within the processing clusters.The load store units prefetch, from the non-coherent memory portion ofexternal memory 170, data associated with the future work units. Foreach core within processing clusters 156, the load store units of thecore may store the prefetched data associated with the WU to beprocessed by the core into a standby segment of the level 1 cacheassociated with the processing core.

FIG. 4 is a block diagram illustrating another example processingcluster 180 including a plurality of programmable processing cores182A-182N. Each of processing clusters 156 of DPU 150 of FIG. 3 may beconfigured in a manner substantially similar to that shown in FIG. 4. Inthe example of FIG. 4, processing cluster 180 includes cores 182A-182N(“cores 182”), a memory unit 183 including a coherent cache memory 184and a non-coherent buffer memory 186, a cluster manager 185 including WUqueue manager 187 for maintaining (e.g., within hardware registers ofprocessing cluster 180) and manipulating WU queues 188, and accelerators189A-189X (“accelerators 189”). Each of cores 182 includes L1 buffercache 198 (i.e., core 182 includes L1 buffer cache 198A and in general,core 182N includes L1 buffer cache 198N). In some examples, clustermanager 185 is alternatively located within central cluster 158, and/orWU queues 188 are alternatively maintained within central cluster 158(e.g., within hardware registers of central cluster 158).

An access node or DPU (such as access nodes 17 of FIG. 1, DPU 130 ofFIG. 2, or DPU 150 of FIG. 3) may support two distinct memory systems: acoherent memory system and a non-coherent buffer memory system. In theexample of FIG. 4, coherent cache memory 184 represents part of thecoherent memory system while non-coherent buffer memory 186 representspart of the non-coherent buffer memory system. Cores 182 may representthe processing cores discussed with respect to DPU 150 of FIG. 3. Cores182 may share non-coherent buffer memory 186. As one example, cores 182may use non-coherent buffer memory 186 for sharing streaming data, suchas network packets.

In general, accelerators 189 perform acceleration for variousdata-processing functions, such as table lookups, matrix multiplication,cryptography, compression, regular expressions, or the like. That is,accelerators 189 may comprise hardware implementations of lookupengines, matrix multipliers, cryptographic engines, compression engines,regular expression interpreters, or the like. For example, accelerators189 may include a lookup engine that performs hash table lookups inhardware to provide a high lookup rate. The lookup engine may be invokedthrough work units from external interfaces and virtual processors ofcores 182, and generates lookup notifications through work units.Accelerators 189 may also include one or more cryptographic units tosupport various cryptographic processes. Accelerators 189 may alsoinclude one or more compression units to perform compression and/ordecompression.

An example process by which a processing cluster 180 processes a workunit is described here. Initially, cluster manager 185 of processingcluster 180 may queue a work unit (WU) in a hardware queue of WU queues188. When cluster manager 185 “pops” the work unit from the hardwarequeue of WU queues 188, cluster manager 185 delivers the work unit toone of accelerators 189, e.g., a lookup engine. The accelerator 189 towhich the work unit is delivered processes the work unit and determinesthat the work unit is to be delivered to one of cores 182 (inparticular, core 182A, in this example) of processing cluster 180. Thus,the one of accelerators 189 forwards the work unit to a local switch ofthe signaling network on the DPU, which forwards the work unit to bequeued in a virtual processor queue of WU queues 188.

As noted above, in accordance with the techniques of this disclosure,one or more of accelerators 189 may be configured to evaluate regularexpressions. A RegEx accelerator of accelerators 189, in accordance withthe techniques of this disclosure, may include a hardware-implementedDFA engine that executes one or more DFAs constructed according totarget regular expressions, i.e., regular expressions to be evaluated aspart of a service. That is, the DFA engine of a RegEx accelerator walksone or more DFA graphs to, effectively, compare an input search stringto one or more regular expressions, to which the DFA graphs correspond,to determine whether the input search string matches any of the regularexpression, as discussed in greater detail below.

After cluster manager 185 pops the work unit from the virtual processorqueue of WU queues 188, cluster manager 185 delivers the work unit via acore interface to core 182A, in this example. An interface unit of core182A then delivers the work unit to one of the virtual processors ofcore 182A.

Core 182A processes the work unit, which may involve accessing data,such as a network packet or storage packet, in non-coherent memory 156Aand/or external memory 170. Core 182A may first look for thecorresponding data in cache 198A, and in the event of a cache miss, mayaccess the data from non-coherent memory 156A and/or external memory170. In some examples, while processing the work unit, core 182A maystore information (i.e., the network packet or data packet) associatedwith the work unit in an active segment of cache 198A. Further, core182A may, while processing the work unit, prefetch data associated witha second work unit into a different, standby segment of cache 198A. Whencore 182A completes processing of the work unit, core 182A initiates (orcauses initiation of) a cache flush for the active segment, and may alsoinitiate prefetching of data associated with a third work unit (to beprocessed later) into that active segment. Core 182A (or a virtualprocessor within core 182A) may then swap the active segment and thestandby segment so that the previous standby segment becomes the activesegment for processing of the next work unit (i.e., the second workunit). Because data associated with the second work unit was prefetchedinto this now active segment, core 182A (or a virtual processor withincore 182A) may be able to more efficiently process the second work unit.Core 182A then outputs corresponding results (possibly including one ormore work unit messages) from performance of the work unit back throughthe interface unit of core 182A.

As described herein, in some example implementations, load store unitswithin memory unit 183 may, concurrent with execution of work units bycores 182 within the processing cluster 180, identify work units thatare enqueued in WU queues 188 for future processing by the cores. Theload store units prefetch, from a non-coherent memory portion ofexternal memory 170, data associated with the future work units andstore the prefetched data associated with the WUs to be processed by thecores into a standby segment of the level 1 cache associated with theparticular processing cores.

FIG. 5 is a block diagram illustrating an example regular expression(RegEx) accelerator 200, in accordance with the techniques of thisdisclosure. RegEx accelerator 200 may correspond to one of accelerators146 of FIG. 2 or one of accelerators 189 of FIG. 4. In this example,RegEx accelerator 200 includes control block 202, on-chip memorydedicated for RegEx accelerator 200, referred to as buffer memory 204,deterministic finite automata (DFA) engines 206, and DFA caches 208,which operates as high-speed on-chip cache memory for caching select DFAarcs. As shown in FIG. 5, RegEx accelerator 200 is also in communicationwith external memory 210. External memory 210 is so named becauseexternal memory 210 is external to RegEx accelerator 200, i.e., offchip, and generally has longer memory access cycles. For example,external memory 210 may correspond to memory unit 134 of FIG. 2 (e.g.,non-coherent buffer memory 138 of FIG. 2), external memory 170 of FIG.3, or non-coherent buffer memory 186 of FIG. 4.

In general, control block 202 represents a processing unit (implementedin circuitry) that controls operation of other components of RegExaccelerator 200. For example, control block 202 may receive work unitsfrom external components (such as processing cores) to traverse a DFA(representing a regular expression) for target input data (e.g., apayload of a packet). In particular, one or more cores of a processingcluster, such as cores 182 of processing cluster 180 in FIG. 4, issue aninstruction to load, and control block 202 loads, a DFA graph (or insome cases, multiple DFA graphs) that was previously compiled from acorresponding regular expression by a compiler. In this way, each DFAgraph generated by the compiler corresponds to at least a portion of aregular expression and is a data structure represents the pattern and/orrule matching criteria set forth within the regular expression. Asdescribed in further detail below, after a compiler compiles regularexpressions into DFA graphs, a loader may allocate data for the DFAgraph to on-chip buffer memory 204 and/or external memory 210, and mayoptimize the structure of the data based on the particular memory towhich the data will be stored when used for stream processing. In someexamples, the loader allocates data for nodes of the DFA graph bytraversing the DFA graph in a breadth-first manner starting from a rootof the DFA graph so as to allocate the nodes of the DFA that are closerto the root first to buffer memory 204 and then to external memory 210once buffer memory 204 is full or a pre-determined amount of buffermemory 204 will be utilized by the portion of the DFA graph allocated tothe buffer memory.

After compilation, the loader stores data representing the DFA graphinitially in external memory 210 or a different computer-readablestorage medium for loading when needed for stream processing. In someexamples, control block 202 may receive work units includinginstructions to retrieve at least a portion of a DFA graph from externalmemory 210 allocated and structurally arranged for buffer memory 204 bythe loader following compilation of the regular expression. In response,control block 202 may retrieve the designated portion of the DFA graphfrom external memory 210 and store the portion of the DFA graph to oneor more of buffer memory 204, and in some cases may preload certainnodes into high-speed, on-chip DFA caches 208, which may operate as L1caches. Likewise, after one or more searches have been conducted,control block 202 may receive work units including instructions to clearone or more of DFA caches 208 and/or unload portions of DFAs from buffermemory 204. Furthermore, control block 202 may receive work unitsincluding instructions to initiate a search, e.g., indicating a payloadto be searched using a loaded DFA graph. In some examples, a single workunit may represent both a command to load a DFA and to perform a searchusing the loaded DFA.

In general, a DFA graph includes a set of nodes directly linked by arcs,where each node in the graph represents a state and each arch representstransitions between states based on criteria specified for therespective arc. Each node of a DFA graph may contain one or more arcsdirectionally linking the node to itself and/or other nodes within theDFA graph.

As further described below, when compiling one or more regularexpressions into one or more DFA graphs, the compiler may generate oneor more of the nodes in a form of a hash table having a set of hashbuckets for storing data indicative of the state transitions representedby the arcs originating from the node. Input, such as symbols withinpayloads of stream data, are hashed to hash buckets to determine whetherthe input results in a state transition for the given node. Moreover,the compiler may arrange each hash bucket in the form of a set of slots,and data representative of the arcs of the DFA may be stored in theslots of hash buckets. Further, when generating the DFA graph, thecompiler may control and arrange the number of slots each hash bucketfor a given node based on the target memory designated to store the nodewhen the DFA graph is to be applied. For example, each of buffer memory204 and external memory 210 are configured so as to allow a certainamount of memory to be read in a single access, generally referred toherein as a memory slice. A memory slice may, for example, represent acertain number of memory-aligned bytes in buffer memory 204 or a certainnumber of aligned bytes in external memory 210. Moreover, the number ofbytes of buffer memory 204 allocated for memory slices may differ fromthat of external memory 210. In general, memory slices of externalmemory 210 are larger that memory slices of buffer memory 204, such thatbuffer memory 204 generally stores fewer bytes for memory slices thanexternal memory 210. In one example, buffer memory 204 stores memoryslices having 32 bytes of data and is 32-byte aligned, while externalmemory 210 stores memory slices having 64 bytes of data and is 64-bytealigned. As further described below, the compiler may construct theformat and arrangement of the hash table representing a given node in aDFA graph to optimize the node for memory access based on the targetmemory to which the node will be allocated when used by RegExaccelerator 200 for stream processing. For example, the compiler maycontrol the number of slots within each row of the hash table (i.e.,each hash bucket) so that the row occupies a single or multiple of thememory slice for the memory selected by the compiler for storing thenode when the DFA graph is loaded for use, thereby decreasing memoryaccess times when applying the DFA graph for stream processing.

In this way, the compiler allocates a node with more arcs than thenumber of slots per slice to a power of 2 slices using one or more hashfunctions, with these nodes being referred to herein as HASH nodes.Labels for arcs from a node of the DFA graph may act as keys to the hashfunctions, such that DFA engines 206 execute the hash functions on thelabels of the arcs. In other words, the hash functions may map thelabels for the arcs to respective hash buckets, which may correspond toslots within one or more additional memory slices for a current node ofthe DFA graph storing, the slots of the additional memory slices storingadditional arcs for the current node. Control block 202 or one of DFAengines 206 may locate data for the nodes of the DFA graph using a modevalue describing in which way the node is allocated. Thus, control block202 or one of DFA engines 206 stores retrieves data describing the modevalue for a particular node.

In yet another example, the compiler may generate some of the nodes notas hash tables but instead in a more compact form such that the node canbe stored within a single memory slice of the memory designated forstorage of the node. The compiler, for example, may determine that anode of a DFA graph has fewer arcs than the number of slots per memoryslice and may then construct the node so as to occupy a single slice. Insome examples, the compiler may allocate the node for storage in thetarget memory in a manner that overlays the node on unused slots of amemory slice that is designated to store a hash bucket for a hash node.In other words, these nodes (referred to herein as a ‘fill node’) areconstructed and allocated to memory addresses of the targeted memory soas to occupy unused slots of hash buckets of HASH nodes.

In some examples, a first subset of the nodes of a DFA graph may bestored in buffer memory 204 and transition to the remaining nodes of theDFA graph stored in external memory 201. The data representative of thearcs may, in these examples, include a locator value that indicateswhether data for a respective subsequent node of the DFA graph (pointedto by the arc) is stored in buffer memory 204 or external memory 210. Inthis manner, DFA engines 206 may use the locator value to retrieve datafor the respective subsequent arc from either buffer memory 204 orexternal memory 210, and store the retrieved data in, e.g., a respectiveone of DFA caches 208. In some examples, when the data for thesubsequent node is stored in external memory 210, DFA engines 206 mayretrieve the data for the subsequent node from external memory 210 andstore this retrieved data to buffer memory 204.

Each of DFA engines 206 include one or more hardware threads configuredto execute respective search processes according to a DFA graph. Each ofthe threads may include, for example, one or more respective memories(e.g., registers, caches, or the like) for storing a current node of acorresponding DFA graph and a current position of a payload data beinginspected. That is, the threads may store data representing a currentnode locator and a payload offset. The current node locator maycorrespond to a value stored by a thread including a memory type (e.g.,buffer memory 204 or external memory 210), address, and mode (size andlayout) of the current node.

DFA engines 206 also include respective processing units for comparing acurrent symbol of the payload data to labels for arcs from the currentnode of the DFA graph. The threads of each of DFA engines 206 may sharea common processing unit, or the threads may each include acorresponding processing unit. In general, the processing unitdetermines a node to which to transition from the current node (i.e.,the node to which the arc having a label matching the current symbol ofthe payload data points). More particularly, given a current nodelocator and an input byte (i.e., the value of a current symbol of thepayload data), the processing unit reads the node from the memorylocation indicated by the current node locator and determines an arc ofthe node (if any) having a label that is the same as the input byte. Ifthe processing unit finds such an arc, the processing unit provides thenext node locator for the next input byte. On the other hand, if no sucharc is found, the processing unit may reinitialize the next node locatorto the start node (i.e., a root of the DFA graph).

The processing unit or the thread of the corresponding one of DFAengines 206 may then update the current node locator and the payloadoffset. The processing unit may continue this evaluation until eitherthe entire set of payload data has been examined without finding amatch, or a resulting node of the DFA graph is a matching node. Inresponse to reaching a matching node, the thread of the one of DFAengines 206 may return data indicating that a match has been identified.

In some examples, before evaluating payload data, DFA engines 206 maypreload at least a portion of a DFA graph into buffer memory 204 fromexternal memory 210 or a different computer-readable medium based on thememory allocation specified by the compiler for each nodes. Additionallyor alternatively, DFA engines 206 may preload a portion of the DFA graphinto memory of a thread of the one of DFA engines 206. In particular,DFA engines 206 may be configured to receive a DFA LOAD work unit,including instructions to direct the DFA engine to load at least aportion of a DFA graph (e.g., a root of the DFA graph, and/or otherportions of the DFA graph) into buffer memory 204 and/or memory of oneof the threads of the DFA engines 206. The at least portion of the DFAgraph may include a root node of the DFA graph and/or data representingone or more nodes and/or arcs of the nodes of the DFA graph. Likewise,DFA engines 206 may be configured to unload a loaded portion of a DFAgraph from the thread memory and/or from buffer memory 204, e.g., inresponse to a DFA UNLOAD work unit. The DFA UNLOAD work unit may includeinstructions indicating that one or more loaded arcs of a DFA graph areto be removed from thread memory and/or buffer memory 204, and/or tounlock and clear a root buffer for a DFA graph from the thread memoryand/or buffer memory 204.

To perform a search, DFA engines 206 may receive a DFA SEARCH work unitincluding instructions to cause DFA engines 206 to select an idle threadof DFA engines 206 to be used to search payload data against a DFAgraph, at least a portion of which may have been previously loaded inresponse to a DFA LOAD work unit. To perform the search, DFA engines 206may provide to the idle thread: data representing locations of the DFAgraph (including a root of the graph, a base address of a portion of theDFA graph loaded into buffer memory 204, and a base address of a portionof the DFA graph in external memory 210), a node from which to start theDFA graph traversal, addresses of payload buffers to be processed in awork unit stack frame, and an address and size of a result buffer in thework unit stack frame.

Accordingly, as discussed above, a thread and a processing unit of oneof DFA engines 206 may perform a search in response to a DFA SEARCH workunit. In particular, the processing unit may retrieve a current symbolfrom payload data of the work unit stack frame, as indicated by the DFASEARCH work unit, and ultimately output an indication of whether a matchoccurred to the result buffer in the work unit stack frame.

An example search algorithm is described below. Inputs to the algorithminclude a location of a root of a DFA graph (root_node_locator),addresses of the DFA graph in buffer memory 204 and external memory 210,a starting node for the traversal of the DFA graph, payload bytes usedto traverse the graph, and an address to which to write matchingresults. Starting from the first byte in the first payload buffer andthe start node locator, a DFA thread of one of DFA engines 206 matcheseach payload byte (cur_label:=payload[cur_offset]) with an arc to a DFAnode (cur_node:=dfa_graph[cur_node_locator]). The example matchingalgorithm, which may be performed by the processing unit of the one ofDFA engines 206, is as follows:

-   -   1. If the node at cur_node_locator contains an arc that maps        cur_label to a next_node_locator, then:        -   a. cur_offset←cur_offset+1        -   b. cur_node_locator←next_node_locator    -   2. If the node at cur_node_locator does NOT contain an arc for        cur_label, then:        -   a. cur_offset remains the same        -   b. cur_node_locator←root_node_locator.

After step 2 above, the processing unit matches the current payload byteto the arcs from the root node. In some examples, this match may beperformed in parallel with another byte of the payload, e.g., if theroot is preloaded into one of DFA cache memories 208 associated with thethread.

The following example algorithm describes one process for writing datato a result buffer. In this example, a DFA thread may add a result entryto the result buffer. If the current node arc has a MATCH attributeindicating that the subsequent node from this arc is a MATCH, the DFAthread adds data representing the current payload offset and next nodelocator to the result buffer.

The DFA thread may continue to match consecutive payload bytes withsuccessive DFA nodes until either the last payload byte is processed,the result buffer becomes full, or a memory error is detected.Ultimately, DFA engines 206 may generate a return work unit includingdata indicating that the search has resulted in a match (but not thelast match), the search has resulted in a match and it is the lastmatch, that the result buffer is full, or an error code if an erroroccurred during the search. RegEx accelerator 200 may send the returnwork unit to the unit that issued the DFA SEARCH work unit.

Each of DFA engines 206 correspond to respective, private DFA cachememories 208. DFA cache memories 208 may serve two purposes: cache arcdata (e.g., recently traversed arcs from a node for which data is storedin external memory 210), and cache root buffer data (e.g., cachingpre-loaded root data from external memory 210 for parallel lookups inresponse to arc cache misses). An entire one of DFA cache memories 208may be used as an arc cache, where each cache line holds one node arc.DFA engines 206 may load these node arcs and evict these node arcsdynamically in the arc cache when they are accessed and traversed by arespective DFA thread.

In addition, DFA engines 206 may use part of the respective one of DFAcache memories 208 as a software-managed root buffer, where each cacheline may hold two preloaded root arcs. If a DFA graph has its root datain external memory 210, DFA engines 206 may first need to receive a DFALOAD work unit to preload the root arcs into the root buffer beforeperforming a search using the DFA graph. Eventually, DFA engines 206 mayalso need to receive a DFA UNLOAD work unit to unload the DFA arcs, oncethe DFA graph is no longer in use.

FIG. 6A is a conceptual diagram illustrating an example DFA graph 220.As noted above, a DFA graph represents a set of one or morecorresponding regular expressions. That is, as noted above, a compilergenerates the DFA graph from the corresponding one or more regularexpressions. In general, DFA graphs include nodes (representing states)with arcs (directed links representing transitions) pointing from onenode to another. In the example of FIG. 6A, DFA graph 220 includes nodes222-244, with arcs having labels between the various nodes. For example,node 222 has an arc to node 224 with label ‘a.’ As another example, node224 has an arc to node 232 with label ‘b.’ Furthermore, node 222 of DFAgraph 220 represents a start node, as indicated by the label “START.”Likewise, nodes 240, 242, and 244 of DFA graph 220 represent examplematch nodes that match a corresponding regular expression. In theexample of FIG. 6A, strings “aaa”, “abd”, and “cb” result in matches.

Although not shown in this example, other terminal and non-matchingnodes (e.g., nodes 244, 234, 236, and 238) may have arcs pointing backto node 222. Alternatively, no such arcs may be needed, but instead, aprocessing unit of a DFA engine may be configured to transition to node222 if no matching arc can be found for a given input byte from acurrent node. For example, if at node 232, an input value of ‘a’ isevaluated, the processing unit may transition back to node 222.Furthermore, various arcs may be included that interconnect the nodes ofDFA graph 220. For example, node 226 may include an arc that points tonode 224.

FIG. 6B is a flowchart illustrating an example method for allocatingdata representing nodes of a DFA graph between buffer memory 204 andexternal memory 210 of RegEx accelerator 200 of FIG. 5 and generatingthe data based on the allocation to optimize or otherwise improve memoryaccess and utilization, in accordance with the techniques of thisdisclosure. The method of FIG. 6B is described as being performed by acompiler and a loader executed by one or more processing cores 182 ofprocessing cluster 180 of FIG. 4.

Initially, the compiler receives a set of one or more regularexpressions (320). The compiler compiles the regular expression into aDFA graph (322), such as DFA graph 220 of FIG. 6A.

The loader then allocates data for the nodes and arcs to slots of memoryslices in buffer memory 204 and/or external memory 210, includingassigning the base memory address for the node and specifying the slotsto be used to store the arc data for the node. Moreover, based on thenumber of arcs, the loader determines whether the data representing thenode will be arranged as HASH node have a plurality of hash buckets oras a FILL node occupying a portion of a single memory slice. In general,when constructing and allocating the nodes to memory by selecting thebase address for each node, the loader traverses the DFA graph andallocates the larger HASH nodes first and then allocates the FILL nodesto occupy unused slots within the hash buckets of the HASH nodes. Theloader allocates the data for the nodes and arcs to buffer memory 204until buffer memory 204 is full, and then transitions the allocation toexternal memory 210. As noted above, buffer memory 204 and externalmemory 210 may have memory slices of different sizes, e.g., 32 bytes and64 bytes, respectively, and the loader generates and structures the HASHnodes and, in particular, the size of the hash buckets, based on theparticular memory to which the node is being allocated.

During the process, the loader selects a next node (e.g., initially, thestart or root node) of the DFA graph and traverses the graph in breadthfirst order so nodes closer to the root tend to be allocated to on-chipbuffer memory (324). For each node, the loader determines whether thenumber of arcs from the node is greater than the number of slots in amemory slice of the current memory (buffer memory 204 or external memory210) and controls the structure of the node based on the number of arcsand the current memory to which the node is to be allocated (326).

If the number of arcs is greater than the number of slots in a memoryslice (“YES” branch of 326), the loader generates the data representingthe node as a hash table and stores a mode value for the current nodeindicating that arcs for the current node are stored in hash bucketsaccording to one or more hash functions (328). Furthermore, the loaderexecutes one or more hash functions (e.g., four hash functions) todetermine the hash buckets to which to allocate arc data, where the sizeof the buckets are controlled to have a specific number of slots suchthat each hash bucket occupies a single or multiple of a memory slice(330). The loader then allocates data for the arcs of the current nodeto the resulting hash buckets (332).

On the other hand, if the number of arcs is not greater (i.e., less thanor equal to) the number of slots in a memory slice (“NO” branch of 326),the loader generates the data representing the node in the form of aFILL node by determining open slots among previously allocated HASH node(334) to which to allocate arc data for the current node. That is, theloader determines a memory slice that may include a number of previouslyfilled slots for a previous hash node, but also at least enoughunallocated slots to store data for arcs of the current node. The loaderthen allocates the arcs of the current node to the open slots of thememory slice (336). The loader then stores a mode value indicating thatthe current node is allocated as a FILL node, as well as positions forarcs of the current node (338).

Tables 1 and 2 below provide examples of different fill modes for a DFAgraph node, depending on whether the arcs for the node are allocated tobuffer memory 204 or external memory 210. In particular, Table 1represents an example set of fill modes for buffer memory 204 (assumingsix slots per 32-byte slice of buffer memory 204), while Table 2represents an example set of fill modes for external memory 210(assuming ten slots per 64-byte slice of external memory 210). In Tables1 and 2, X's represent slots to which arc data is allocated for a modeindicated by the mode value

TABLE 1 SLOT NUMBERS MODE 5 4 3 2 1 0 1 X 2 X 3 X 4 X 5 X X 6 X X 7 X X8 X X X 9 X X X 10 X X X 11 X X X X 12 X X X X 13 X X X X X 14 X X X X XX

TABLE 2 SLOT NUMBERS MODE 9 8 7 6 5 4 3 2 1 0 1 X 2 X 3 X 4 X 5 X 6 X 7X X 8 X X 9 X X 10 X X 11 X X 12 X X 13 X X 14 X X 15 X X 16 X X X 17 XX X 18 X X X 19 X X X 20 X X X 21 X X X X 22 X X X X 23 X X X X X 24 X XX X X 25 X X X X X X 26 X X X X X X X 27 X X X X X X X X 28 X X X X X XX X X 29 X X X X X X X X X X

After allocating the arc data for the node according as either a HASHnode or a FILL node as discussed above, the loader moves to the nextnode of the DFA graph (in breadth first order) (324) and proceeds toallocate the data for that node, until data for arcs of all nodes hasbeen allocated. As noted above, in some examples, the loader traversesthe DFA graph in breath-first manner and allocates HASH nodes during thetraversal while setting aside FILL nodes for subsequent allocationwithin the unused slots of the HASH nodes. It should be understood that“allocated” refers to determining the bases addresses where the nodesand arc data are to be subsequently loaded and stored including whetherthe data for the arcs of the node is to be stored in external memory 210or read into buffer memory 204, e.g., in response to a DFA LOAD WU. Theloader may store such data to external memory 210 or othercomputer-readable medium.

FIG. 6C is a conceptual diagram illustrating an example set of memoryslices 350 of buffer memory 204. In particular, the example set ofmemory slices 350 includes memory slices 352A-352E (memory slices 352).In this example, each of memory slices 352 includes six slots, althoughit should be understood that in other examples, other numbers of slotsmay be used. Moreover, although discussed with respect to the particularexample of buffer memory 304 having 32-byte memory slices with sixslots, similar techniques may apply to memory slices of external memory310, which may be 64 byte memory slices with ten slots.

FIG. 6C also depicts an example node 354 of a DFA graph. In thisexample, arcs for node 354 is generated by the compiler to be stored inhash mode, such that node 354 may be viewed as a HASH node having anumber of hash buckets mapped to respective memory slices 352. That is,the compiler selects and utilizes hash function 356A to map label datafor node 354 to memory slice 352B, and utilizes hash function 356B tomap the label data for node 354 to memory slice 352C. Data for arcs ofnode 354 is stored in slots of memory slices 352B, 352C as representedby diagonal hashing. As seen in this example, arc data of node 354 thathashes to a hash bucket occupying memory slice 352B only utilizes fourof the six slots.

FIG. 6C further depicts another example node 358 of the DFA graph. Inthis example, the compiler has generated the data for representing node358 as a FILL node, where the arcs for node 358 are stored in fill modewithin unused slots of the overall hash table allocated for hash node354. In particular, with respect to the example of Table 1 above, arcsfor node 358 are stored according to fill mode 7 so as to occupy slots 0and 1 within memory slice 352B of on-chip buffer memory. That is, arcdata for node 358 is stored in slots of memory slice 352B, asrepresented by vertical hashing, which are unused slots of a hash bucketof HASH node 354 mapped to memory slice 352.

In this manner, the compiler or loader may allocate slots of memoryslices of, e.g., buffer memory 304, to arcs for nodes of a DFA graph.The compiler may allocate slots of memory slices for external memory 310in a similar fashion, although with the recognition that the memoryslices of external memory 310 may be larger than the memory slices ofbuffer memory 304.

FIGS. 7A through 7C are flowcharts illustrating example regularexpression operations performed in accordance with the techniques ofthis disclosure. The operations of the methods of FIGS. 7A through 7Cmay be performed by, e.g., the components of RegEx accelerator 200 asdiscussed below, or other devices in other examples. Furthermore, theoperations of the methods of FIGS. 7A through 7C may be performed in adifferent order or with fewer operations than what is shown in FIGS. 7Athrough 7C.

As shown in FIG. 7A, initially, RegEx accelerator 200 receives a DFALOAD work unit (WU) (250). As discussed above, the DFA LOAD work unitmay specify a DFA graph to be loaded into, e.g., buffer memory 204and/or one of DFA caches 208. The DFA LOAD work unit may also specify aportion of the DFA graph to be loaded, e.g., a root node of the DFAgraph and/or one or more other nodes and corresponding arcs. In responseto the DFA LOAD work unit, control block 202 may cause one of DFAengines 206 to load the indicated portion of the indicated DFA graph(252), e.g., into buffer memory 204 and/or into a corresponding one ofDFA caches 208. In this manner, the one of DFA engines 206 stores atleast a portion of a DFA graph to a DFA buffer of a memory, the DFAgraph comprising a plurality of nodes, each of the nodes having zero ormore arcs each including a respective label and pointing to a respectivesubsequent node of the plurality of nodes, at least one of the pluralityof nodes comprising a match node.

Next, as shown in FIG. 7B, after having previously loaded the portion ofthe DFA graph, RegEx accelerator 200 receives a DFA SEARCH work unit(254). In some examples, RegEx accelerator 200 may receive a single workunit representing both load and search instructions. The DFA SEARCH workunit, as discussed above, specifies payload data to be compared to theDFA graph. In response to receiving the DFA SEARCH work unit, controlblock 202 directs the work unit to one of DFA engines 206, which assignsthe search to an idle hardware thread thereof (256). The one of DFAengines 206 also initializes the DFA thread (258). For example, usingdata of the DFA SEARCH work unit, the one of DFA engines 206 sets avalue of a current node memory for the thread to represent a currentnode (e.g., a start node) of the DFA graph and a value of a payloadoffset to represent a current byte of the payload (e.g., a startingsymbol of the payload). The one of DFA engines 206 may further maintaindata representing a location of a result buffer to which output data isto be written as a result of performing the search.

The DFA thread of the one of DFA engines 206 may then search symbols ofthe payload data against the DFA graph (260). In particular, the DFAthread may compare (or cause a processing unit of the one of DFA engines206 to compare) the current symbol of the payload data to labels of arcsfrom the current node of the DFA graph. If one of the labels is the sameas the current symbol, the DFA thread may update the current node valueto correspond to the node pointed to by the arc having the label that isthe same as the current symbol, and increment the value for the positionof the current symbol, to move to a next symbol in the payload data. Ifnone of the labels match the current symbol, the DFA thread maytransition back to the start node of the DFA graph, and compare thecurrent symbol to the arcs from the start node of the DFA graph. If thestart node has been loaded into one of DFA caches 208, the DFA threadmay perform the comparison of the current symbol of the payload data tothe arcs from the start node in parallel to the comparisons of thecurrent symbol to the arcs from the current node.

In this manner, the DFA thread determines a value of a current nodememory representing a current node of the plurality of nodes in the DFAgraph, a value of a payload offset memory representing a position ofcurrent symbol in a sequence of symbols of payload data, and a label ofone of the arcs of the current node that matches the current symbol.Furthermore, in this manner, the DFA thread updates the value of thecurrent node memory to a value representative of the respectivesubsequent node of the one of the arcs having the label that matches thecurrent symbol. Likewise, in this manner, the DFA thread increments thevalue of the payload offset memory.

In some examples, the DFA thread may determine the label of one of thearcs of the current node that matches the current symbol byspeculatively accessing a cache of RegEx accelerator 200 by accessingthe two or more node representations of the DFA graph that are stored inthe cache in parallel. That is, RegEx accelerator 200 may maintain in amemory, a reference node representation for the at least the portion ofthe DFA graph, and a respective delta node representation for each nodeof the at least the portion of the DFA graph other than the referencenode, each respective delta node representation defining each of thezero or more arcs of a different, corresponding node from the pluralityof nodes that is not defined by the representation of the referencenode. As such, the DFA thread may determine a label of one of the arcsof the current node that matches the current symbol by speculativelyaccessing the two or more node representations in parallel to determinewhether the current symbol matches the label.

The DFA thread may determine whether a match node of the DFA graph hasbeen reached (262). The DFA thread may encounter zero, one, or morematch nodes of the DFA graph, depending on the DFA graph and the payloaddata. In response to reaching a match node (“YES” branch of 262), theDFA thread may output data indicating that a match has occurred (264).In some examples, the DFA thread outputs data for each match that hasoccurred. For example, the DFA thread may write data to the resultbuffer, as discussed above. If no match occurs for the entire payload,in some examples, the DFA thread outputs data indicating that no matchhas occurred, and that the payload has ended. In this manner, inresponse to updating the value of the current node memory to correspondto the match node, the DFA thread outputs an indication that the payloaddata has resulted in a match.

In the example of FIG. 7B, it should be understood that multiplesearches may be performed using the same loaded DFA graph. That is, oneor more DFA threads may perform multiple searches on multiple differentpacket payloads using the same DFA graph after the DFA graph has beenloaded, without unloading the DFA graph between searches. Furthermore,in the example of FIGS. 7A and 7B, it should be understood that, in someinstances, both the load operations of FIG. 7A and the searchesoperations of FIG. 7B are performed concurrently in response to a single“atomic” search command.

Finally, turning to FIG. 7C, in some instances, after performing thesearch or after performing multiple searches according to the operationsof FIG. 7B, one of the DFA engines 206 receive a DFA UNLOAD work unit(266). In response to the DFA UNLOAD work unit, the one of DFA engines206 removes the DFA graph data from the corresponding one of DFA caches208 and/or buffer memory 204 (268). It should be understood that theoperations of FIG. 7C are optional and in some instances the DFA engines206 may never perform the unload operations of FIG. 7B after performingthe search or after performing multiple searches.

FIG. 8 is a flowchart illustrating an example method for traversing aDFA graph from one node to a next node to apply the DFA graph to streamdata in accordance with the techniques of this disclosure. Inparticular, the flowchart of FIG. 8 represents traversing a DFA graphstored as hash nodes and/or fill nodes in a memory of the RegExaccelerator so as to apply the DFA graph to stream data units, such aspackets. The method of FIG. 8 may be performed as part of the methods ofFIG. 7A through 7C discussed above. For example, the method of FIG. 8may be performed iteratively as part of step 260 of FIG. 7B. Again, andfor purposes of example, the method of FIG. 8 is explained with respectto the components of RegEx accelerator 200, although other devices maybe configured to perform this or a similar method.

Initially, it is assumed that a DFA thread of one of DFA engines 206stores a value representing a current node of a DFA graph, as well as acurrent symbol of a payload being evaluated. For example, the DFA threadmay maintain a payload offset value representing the position of acurrent symbol in the payload. Accordingly, the DFA thread determines acurrent symbol of the payload, i.e., the value (e.g., byte value) at thepayload offset within the payload (280).

The DFA thread also determines a mode value for the current node of theDFA graph, i.e., whether the structure of the node and the memory layoutand allocation for the node's arc data is a FILL node or a HASH node(282). In general, the mode value may indicate that the current node iseither in a fill mode (i.e., all arcs from the current node are storedin a single memory slice) or a hash mode (i.e., one or more arcs fromthe current node are stored in slots of one or more additional memoryslices). In the case that the mode value indicates that the current nodeis in fill mode (“FILL MODE” branch from 282), the DFA thread comparesthe current symbol to arc labels of arcs in slots of the memory slice(284).

The DFA thread alternatively compares the current symbol to arc labelsof arcs in the slots of the memory slice in the case that the mode valueindicates that the current node is in hash mode (“HASH MODE” branch from282). That is, the DFA thread determines one or more slots of one ormore additional memory slices using one or more hash functions (e.g.,four hash functions) in this case. In particular, the DFA threadexecutes the one or more hash functions on the current symbol value(288), and the buckets resulting from execution of the hash functionscorrespond to slots of the one or more additional memory slices in whichadditional arc data for the current node is stored. Accordingly, the DFAthread also compares labels of the arc data in the slots of the one ormore additional memory slices to the current symbol of the payload(290).

In either case, the DFA thread determines one of the arcs having a labelthat matches the current symbol of the payload. In response to determinethe one of the arcs having the label that matches the current symbol,the DFA thread updates the current node of the DFA graph to the node towhich the one of the arcs points (292) and increments the payload offset(294) to update the current symbol of the payload.

FIG. 9 is a block diagram illustrating an example DFA engine 300. DFAengine 300 may correspond to one of DFA engines 206 of FIG. 5. In thisexample, DFA engine 300 includes hardware DFA threads 302A-302C (DFAthreads 302). Each of DFA threads 302 includes respective current nodevalues 304A-304C (current node values 304) and respective payloadoffsets 306A-306C (payload offsets 306). In addition, DFA engine 300includes processing unit 310.

As discussed above, DFA threads 302 generally maintain values ofrespective current nodes 304 and payload offsets 306 for a currentsearch process. DFA thread 302A, for example, may store datarepresenting a current node of a DFA graph as current node 304A, and aposition of a current symbol of payload data being compared to the DFAgraph as payload offset 306A. DFA thread 302A may then cause processingunit 310 to compare the value of the current symbol indicated by payloadoffset 306A to labels of arcs of the node represented by current node304A. If processing unit 310 determines that a label of one of the arcsmatches the value of the current symbol, DFA thread 302A may incrementpayload offset 306A and update the value of current node 304A to a valuerepresenting a node pointed to by the arc. On the other hand, ifprocessing unit 310 determines that none of the labels of the arcsmatches the value of the current symbol, DFA thread 302A may update thevalue of current node 304A to a value representing a start node of theDFA graph. If current node 304A correspond to a node of the DFA graphconfigured as a match node, DFA thread 302A and/or processing unit 310may output data indicating that match has occurred.

As noted above, DFA engine 300 may be included in RegEx accelerator 200,which may be included in a processing device, such as one of accessnodes 17 (FIG. 1), DPU 130 (FIG. 2), or DPU 150 (FIG. 3). Accordingly,these processing devices represent examples of a processing deviceincluding a memory including a deterministic finite automata (DFA)buffer configured to store at least a portion of a DFA graph, the DFAgraph comprising a plurality of nodes, each of the nodes having zero ormore arcs each including a respective label and pointing to a respectivesubsequent node of the plurality of nodes, at least one of the pluralityof nodes comprising a match node. The processing device also includes aDFA engine implemented in circuitry, the DFA engine comprising one ormore DFA threads implemented in circuitry, each of the DFA threadscomprising a current node memory storing a value representing a currentnode of the plurality of nodes in the DFA graph, and a payload offsetmemory storing a value representing a position of current symbol in asequence of symbols of payload data. The DFA engine further includes aprocessing unit configured to determine a label of one of the arcs ofthe current node that matches the current symbol, update the value ofthe current node memory to a value representative of the respectivesubsequent node of the one of the arcs having the label that matches thecurrent symbol, and increment the value of the payload offset memory. Inresponse to updating the value of the current node memory to correspondto the match node, the DFA engine is configured to output an indicationthat the payload data has resulted in a match.

FIGS. 10A and 10B are conceptual diagrams illustrating noderepresentations of a DFA graph, in accordance with the techniques ofthis disclosure. FIG. 10C is a conceptual diagram illustrating anexample multi-port cache of an example RegEx accelerator, in accordancewith the techniques of this disclosure. FIGS. 10D and 10E are conceptualdiagramed illustrating logical views of a cache of an example RegExaccelerator, in accordance with the techniques of this disclosure. FIGS.10A through 10E are described in the context of RegEx accelerator 200 ofFIG. 5 and DFA graph 220 of FIG. 6A.

In the example of FIG. 10A, a processing unit of one of DFA engines 206may maintain a respective node representation of each of the nodes ofDFA graph 220, or a subset thereof. For example, FIG. 10A shows arepresentation of DFA nodes 222, 224, and 232. For each possible payloadlabel, each node representation includes a corresponding arc exitingthat node with that label, whether an arc actually exists in the DFAgraph or not. In cases where no arc exists for a particular symbol, thenode representation may include an arc back to a start or root node ofDFA graph 220. In other cases where no arc exists for a particularsymbol, the node representation may omit any arc value and instead RegExaccelerator 200 may consider a lack of arc value as an implicitinstruction to return back to a start or root node of DFA graph 220.

Node 222 is a start node, also referred to as a root node, of DFA graph220. The representation of node 222 of FIG. 10A includes a mapping, forevery potential symbol, to a corresponding arc exiting node 222,including information about an arc to node 224 with label ‘a’, an arc tonode 226 with label and an arc to node 222 with a label ‘c’. Therepresentation of node 224 of FIG. 10A includes a mapping, for everypotential symbol, to a corresponding arc exiting node 224 includinginformation about an arc to node 230 with label ‘a’, an arc to node 232with label ‘b’, and an arc to node 234 with a label ‘c’. Lastly, therepresentation of node 232 of FIG. 10A includes a mapping, for everypotential symbol, to a corresponding arc exiting node 232, includinginformation about a single arc to node 244. In the example of FIG. 10A,where any node from DFA graph 220 does not include an arc with a labelmatching a potential symbol, the corresponding node representation mapsthat potential symbol with a back to the root or reference node (e.g.,node 222). In the example of FIG. 10A, string “abd” results in a matchdefined by node 244.

In operation, when evaluating string “abd”, a processing unit of one ofDFA engines 206 may evaluate each symbol (e.g., byte) in a payload“abd”. Initially, the processing unit, may start with noderepresentation 222 (e.g., a current node) and determine, from noderepresentation 222, that a current symbol ‘a’, as defined by the noderepresentation of node 222, maps to an arc to node 224. In response todetermining that, for node 222, the symbol ‘a’ is assigned to an arc tonode 224, the processing unit next uses node representation 224 toprocess symbol ‘b’ from the payload. The processing unit may determinethat node representation 224 assigns the symbol ‘b’ to an arc to node232. In response to determining that, for node 224, the symbol ‘b isassigned to an arc to node 232, the processing unit next evaluatessymbol ‘d’ from the payload. The processing unit may determine, thatnode representation 232 assigns symbol to an arc to node 244. Inresponse to determining that the symbol ‘d’ is assigned to an arc tonode 244, the processing unit may output an indication of a match.

Now turning to FIG. 10B, in some examples, rather than maintaining anentire representation of each of the nodes of DFA graph 220, or a subsetthereof, a processing unit of one of DFA engines 206 may perform “rootoptimization techniques” that minimize an amount of memory, inparticular the amount of cache, that is consumed by the noderepresentations of the nodes of DFA graph 220. That is, to reduce theamount of cache taken up by the arcs of the node representations of thenodes of DFA graph 220, the processing unit of one of DFA engines 206may cache a complete representation (e.g., all arcs) of a root node ofDFA graph 220 while only caching a partial representation (e.g., onlysome arcs) of each of the other (e.g., non-root) nodes of DFA graph 220,or a subset thereof, into one of DFA caches 208.

While described primarily as being a representation of a “root node”, insome examples, RegEx accelerator 200 utilizes one or morerepresentations of “reference nodes” to evaluate a regular expression inview of a DFA graph. That is, a reference node, may be some othernon-root node of a DFA graph. A representation of a reference node maydefine any child node arcs that are the same as arcs of the referencenode. For example, a representation of node 224 may be a reference noderepresentation to nodes 230, 232, and 234. In some cases, a referencenode representation includes an arc for every possible payload symbol.In other cases, the reference node representation only includes arcs forevery possible payload symbol that do not return back to the referencenode; instead, such arcs are considered to be implicitly defined by thereference node representation.

Each partial representation of a node (also sometimes referred to as a“delta node”) may specify each arc that is not already specified orimplied by the representation of the root node. In other words, aprocessing unit of one of DFA engines 206 may store in one of DFA caches208, a root node representation for DFA graph 220 and a respective deltanode representation for each node of DFA graph 220, other than the rootnode, where each respective delta node representation defines any arcexiting that node which is not already defined (explicitly orimplicitly) in the representation of the root node. By caching only aroot node representation and a delta node representation for each othernode of DFA graph 220, the processing unit of one of DFA engines 206 mayconsume less cache of one of DFA caches 208 than if the processing unitcached a complete or full respective representation of each of the nodesof DFA graph 220, or a subset thereof, into one of DFA caches 208, asshown in FIG. 10A.

For example, FIG. 10B shows a representation of node 222, that issimilar to the representation of node 222 of FIG. 10A, however in FIG.10B, the arcs defined by the representation of node 222 are cached inone of DFA caches 208. The cached arcs of root node 222 of FIG. 10Bincludes an arc specified for every possible payload symbol except forany potential symbols that do not correspond to a label of an arcexiting root node 222 or for any symbols that define an arc that returnsto the root node 222; in such a case, the symbol is left undefined byroot node 222, thereby implying an arc back to root node 222.

In contrast to the example of FIG. 10A, the representations of nodes 224and 232 of FIG. 10B are mere partial representations or delta noderepresentations. Each of the delta node representations of nodes 224 and232 includes less information (e.g., fewer arcs) than the correspondingnode representations of nodes 224 and 232 of FIG. 10A. Each of the deltanode representations of nodes 224 and 232 only includes informationabout arcs that are different than the arcs already defined (explicitlyor implicitly) by the node representation of root node 222.

For example, the node representation of node 224 that is cached in oneof DFA caches 208 only includes information about arcs exiting node 224,such as information about an arc to node 230 with label ‘a’, an arc tonode 232 that matches symbol and an arc to node 234 with label ‘c’.Likewise, the node representation of node 232 only includes informationabout an arc to node 244 with label ‘d’. Unlike in the example of FIG.10A, any potential symbol that does not correspond to a label of an arcexiting node 224 or exiting node 232 is left undefined by the delta noderepresentations for nodes 224 and 232. In this way, a processing unit ofone of DFA engines 206 may, when evaluating a payload symbol that isundefined by an arc of a delta node representation, rely on therepresentation of root node 222 to evaluate the symbol.

In one example, a processing unit of one of DFA engines 206 may evaluatepayload “abd” using the representations of FIG. 10B. When evaluatingpayload “abd”, the processing unit of one of DFA engines 206 maydetermine that symbol ‘a’, as defined by the representation of root node222, maps to an arc to node 224. In response to determining that, fornode 222, the symbol ‘a’ is assigned as the label of an arc to node 224,the processing unit may next determine the arc, as defined by therepresentation of node 224, that has a label corresponding to symbol‘b’. The processing unit may determine that an arc from node 224 to node232 has a label ‘b’. Next, the processing unit may determine the arcdefined by the representation of node 232 that has a label correspondingto symbol ‘d’. In response to determining that the label correspondingto symbol ‘d’ is assigned to an arc between node 232 and node 244, theprocessing unit may consume the current symbol and output an indicationof a match.

In another example, a processing unit of one of DFA engines 206 mayevaluate payload “abc” which does not result in a match associated withDFA graph 220. A processing unit of one of DFA engines 206 may determinethe symbol ‘a’, as defined by the representation of root node 222, mapsto an arc to node 224 symbol ‘a’. In response to determining that, fornode 222, the symbol ‘a’ is assigned to an arc to node 224, theprocessing unit goes on to determine the arc defined by the delta noderepresentation of node 224 that corresponds to symbol ‘b’. Theprocessing unit may determine that the symbol ‘b’ is mapped to an arc tonode 232. However, the processing unit may determine that the delta noderepresentation of node 232 does not define an arc for the symbol ‘c’. Inresponse to determining that the symbol ‘c’ is undefined by the deltanode representation of node 232, the processing unit may check the rootnode representation of node 222 to determine whether the root noderepresentation maps an arc to the symbol ‘c’. The processing unit maydetermine that symbol ‘c’ is also undefined by the root noderepresentation for node 222, and may conclude that therefore, the rootnode representation for node 222 implicitly maps symbol ‘c’ to itselfand may therefore output an indication of a no match for payload “abc”.

By storing only a root or reference node representation and one or moredelta node representations, the processing unit of one of DFA engines206 may use less memory than if the processing unit stores a completenode representation for each node in DFA graph 220. However, a drawbackof storing arcs of delta node representations only is that in caseswhere a current symbol is undefined by a delta node representation, theprocessing unit may be required to access memory twice. The processingunit may first accesses DFA caches 208 or memory to determine whether acurrent symbol is defined by a particular delta node representation. Incases where the current symbol is undefined by the particular delta noderepresentation, the processing unit may perform a second access of DFAcaches 208 or other memory, time to determine the root node definitionfor the current symbol. The second access may occur before or after thefirst access. In other examples, the second access may occurconcurrently with the first access.

That is, rather than evaluating a delta node representation and a rootnode representation, sequentially (in any order), a processing unit ofDFA engines 206 may instead perform speculative root node access, orsimply “speculative access” techniques to evaluate a delta and root noderepresentation in parallel. To match a current symbol, a processing unitof DFA engines 206 may speculatively access memory to match a currentsymbol by simultaneously (or nearly simultaneously) accessing a rootnode representation at the same time or nearly the same time that theprocessing unit accesses and a delta node representation. By accessingroot and delta node representations at the same time (e.g., in parallel)as opposed to sequentially, a processing unit may evaluate a regularexpression faster (e.g., in fewer clock cycles) than evaluating theregular expression through multiple, sequential accesses of the deltanode and then the root node representation. In this way, if a currentsymbol is undefined by a particular delta node representation, theprocessing unit need not also access DFA cache 208 to determine the rootnode definition for the current symbol; the processing unit may insteadhave already evaluated the current symbol against the root noderepresentation while the processing unit also evaluated the delta noderepresentation.

The example multi-port cache of FIG. 10C may enable a processing unit ofone of DFA engines 206 to perform speculative access, for instance, whenboth the root and delta node representation are stored in cache 208.FIG. 10C is just one example architecture. In other examples, othermemory architectures may be used. For example, multiple single-port ormultiple multi-port memories may be configured as a “multi-port cache”for purposes of performing speculative access.

In the example of FIG. 10C, an example of one of DFA caches 208 is shownas DFA cache 208A. Stored in DFA cache 208A are arcs of a root noderepresentation of DFA graph 220 and arcs of one or more delta noderepresentations of DFA graph 220. DFA cache 208A further includes two ormore input ports 404A and 404B and two or more corresponding outputports 406A and 406B. In some examples, DFA cache 208A includes oneoutput port for every input port. And in some examples, DFA cache 208Aincludes a different number of output ports as compared to input ports.And in some examples, DFA cache 208A is made up of multiple single portmemories or multiple multi-port memories that are configured to beaccessed in a similar way as a single, multiple-port cache. In any case,by having multiple input and output ports 404A, 404B, 406A, and 406B, aprocessing unit of one of DFA engines 206 can simultaneously access DFAcache 208 to evaluate a current symbol against the root noderepresentation in parallel to evaluating the current symbol against adelta node representation.

For example, a processing unit of one of DFA engines 206 may input cachekey 400A into input port 404A to evaluate a current symbol using theroot node representation. At the same time, or nearly the same time(i.e., in parallel), the processing unit of one of DFA engines 206 mayinput cache key 400B into input port 404B to evaluate the current symbolusing the delta node representations. Cache key 400A may specify acurrent node as being the root node and cache key 400B may specify acurrent node as being a non-root node associated with a delta noderepresentation.

The processing unit may obtain result 402A from output port 406A whichidentifies an arc to a subsequent node associated with the currentsymbol, as is defined by the root node representation. At the same time,or nearly the same time that the processing unit obtains result 402A,the processing unit may obtain result 402B from output port 406B whichidentifies an arc to a subsequent node associated with the currentsymbol, as is defined by a delta node representation that maps to key400B.

In some examples, a processing unit of one of DFA engines 206 may beconfigured to speculatively access DFA caches 208 to match a label of anarc to a current symbol of a payload, in response to identifying a rootnode representation in DFA caches 208. That is, speculatively accessingthe delta and root node representations may only realize a performancegain over sequential access, if the root node representation is alreadyloaded in DFA cache 208A. If not loaded into DFA cache 208A, theprocessing unit may be configured to, in response to not identifying theroot node representation in DFA cache 208A, refrain from speculativelyaccessing DFA cache 208A to match the current symbol to an arc label.Instead, the processing unit may be configured to sequentially evaluatea current symbol against the root and delta node representations byaccessing the root node representation in a level two cache or externalmemory to match the label after accessing any of the delta noderepresentations to match the label. In some examples, in response to notidentifying the root node representation in DFA cache 208A, the rootnode representation may be loaded into DFA cache 208A so that subsequentevaluations of DFA graph 220 can be performed using speculative accesstechniques described above.

In some examples, RegEx accelerator 200 may need to simultaneouslysupport multiple DFA graphs. Therefore, RegEx accelerator 200 may needto maintain multiple root node representations and multiple sets ofdelta node representations (i.e., one root node representation and oneset of delta node representations for each graph). Because the size ofDFA cache 208A may not be sufficient to store all the root and deltanode representations of multiple graphs, RegEx accelerator 200 maycreate a high-performance group of graphs and a low-performance ormedium-performance group of graphs. For any graph in thehigh-performance group of graphs, RegEx accelerator 200 may store theroot node representation for that graph in one of DFA caches 208.Storing a root node representation in one of DFA caches 208 may occur inresponse to a software event, at start up, or after failing to identifythe root node representation (e.g., one time or with sufficientfrequency) in one of DFA caches 208 during runtime. For any graph not inthe high-performance group of graphs, RegEx accelerator 200 may storethe root node representation for that low or medium performance graph inan external memory, such as external memory 210 of FIG. 5. Saiddifferently, RegEx accelerator 200 may maintain a hierarchy of memoriesand caches and when a root node representation of a DFA graph is notstored in a cache that is sufficiently high in the memory hierarchy(e.g., in one of DFA caches 208), then RegEx accelerator may refrainfrom performing speculative root node access.

Although FIGS. 10A through 10E have been described above in the contextof RegEx accelerator 200 and/or a processing unit of one of DFA engines206, in some examples, the techniques described herein may beimplemented by an intelligent cache controller 408 of RegEx accelerator200, e.g., as shown in FIG. 10C. That is, DFA cache 208A may include adedicated component, such as cache controller 408, that is configured tomanage what gets stored in DFA cache 208A and when, so as to satisfy theconditions described above.

Turning to FIG. 10D, to further improve performance when evaluating anexpression, RegEx accelerator 200 may cache at least one of: “effectivearcs” or “negative arcs” so that subsequent evaluations can benefit fromwork already performed during previous evaluations.

As used herein, the term “effective arc” refers to an arc that hasrecently been evaluated to be a match to a previous, current symbol anddoes not lead back to a root or reference node. As the processing unitof one of DFA engines 206 evaluates a current symbol against a referencenode or delta node representation, RegEx accelerator 200 may cache, atone of DFA caches 208, information about effective arcs and then rely onthe cached effective arcs to evaluate a subsequent, current symbol. Forexample, while evaluating node RegEx accelerator 200 amy determine thata current symbol ‘b’ matches an arc to node 232. In response, RegExaccelerator 200 may cache information indicating that whenever a currentsymbol is ‘b’, while evaluating node 224, the subsequent node is node232, based on a previous evaluation.

As used herein, the term “negative arc” refers to an arc which does notexist for a label in a current node, and as a result, a root orreference node must be accessed to process a current symbol. As theprocessing unit of one of DFA engines 206 evaluates a current symbolagainst a reference node or delta node representation, RegEx accelerator200 may determine that the current symbol does not match a label of anarc of a current node. In response to identifying an absence of amatching arc of the current node, RegEx accelerator 200 may cache, atone of DFA caches 208, information about the negative arc; i.e., an archaving a label that matches a current symbol and connects the currentnode to a root or reference node. For example, RegEx accelerator 200 maycache information indicating that when a current symbol is ‘b’ whileevaluating node 232, the subsequent node is node 222. Since accessing acurrent node may at times require an external memory access, cachingnegative arcs in this way may enable RegEx accelerator 200 to have areduced latency. That is, RegEx accelerator 200 may determine asubsequent cache hit of a negative arc, and therefore avoid a futureexternal memory access (or wherever a node representations is stored) byinstead, immediately accessing a root or reference node representationfor processing a current symbol.

In some cases, an effective arc or negative arc stored in cache 208 maynot be an actual arc in a current node; instead, an effective arc ornegative arc may be a resulting arc. For example, image current symbol‘x’ does not match any arc in current node ‘N1’. In this case, RegExaccelerator 200 may re-inspect symbol ‘x’ at root or reference node‘RN’. Node ‘RN’ may or may not have arc to third node ‘N2, for symbol‘x’. In cases where node ‘RN’ does have an arc for symbol, ‘x’, RegExaccelerator 200 may store as an effective arc for symbol ‘x’ betweennodes ‘N1’ and ‘N2’. In other words, a transition from node N1 to nodeRN to node N2 for symbol ‘x’ is stored as a resulting arc for symbol ‘x’from node N1 to node N2. As such, RegEx accelerator 200 may avoidaccessing N1 and RN representations completely and instead receive acache hit of the resulting arc. In cases where node ‘RN’ does not havean arc for symbol, ‘x’, RegEx accelerator 200 may instead store aneffective arc for symbol ‘x’ between nodes N1 and RN.

As demonstrated in FIG. 10D, by caching effective and/or negative arcs,a processing unit of one of DFA engines 206 may avoid doing speculativeroot access if during a current symbol evaluation, the processing unitdetermines that a matching arc for the current symbol is already cachedas an effective or negative arc. In other words, when evaluating acurrent symbol against what is cached in one of DFA caches 208, if theone of DFA caches 208 returns a hit to a matching arc, then theprocessing unit need take no further action. However, if the one of DFAcaches 208 fails to return a hit, the processing unit may continue toperform speculative root access against the information stored in theone of DFA caches 208.

FIG. 10D includes views of cache 208 at different points in time,relative to different cache keys derived from a payload. For example,cache 208 at time t0 is empty or cold and includes zero effective andzero negative arcs. At subsequent times, t1-t5, cache 208 includesinformation about the effective or negative arcs that have beentraversed during the subsequent times. The payload being evaluated inFIG. 10D includes the string “ABCABD”.

As shown in FIG. 10D, at time t0, a processing unit of one of DFAengines 206 may evaluate symbol ‘A’ against the root node 222representation (e.g., stored in a cache, external memory, buffer memory,or other memory). In response to determining that the symbol matches anarc from node 222 to node 224, the processing unit stores, as aneffective arc, an indication of an arc with label ‘A’ from node 222 tonode 224 at location x001 of cache 208.

At time t1, the processing unit may evaluate symbol ‘B’ against the node224 representation (e.g., stored in a cache, external memory, buffermemory, or other memory). In addition to evaluating symbol ‘B’ againstthe representation of node 224, the processing unit may further evaluatethe current symbol in view of the effective and negative arcs stored incache 208. In some examples, the processing unit may evaluate a currentsymbol against cached effective and negative arcs prior, concurrent, orsubsequent to evaluating the symbol against a current noderepresentation. In other words, once cache 208 is no longer cold, theprocessing unit may access cache 208 before, while, or after accessingthe node representation. In response to determining that the symbol ‘B’does not match a label of any effective or negative arcs stored in cache208, and further in response to determining that the symbol matches alabel of an arc from node 224 to node 232, the processing unit stores,as an effective arc, an indication of an arc with label ‘B’ from node224 to node 232 at location x002 of cache 208.

At time t2, the processing unit may evaluate symbol ‘C’ against the node232 representation (e.g., stored in a cache, external memory, buffermemory, or other memory). In addition to evaluating symbol ‘C’ againstthe representation of node 232, the processing unit may further evaluatethe current symbol in view of the effective and negative arcs stored incache 208. In response to determining that the symbol ‘C’ does not matcha label of any effective or negative arcs stored in cache 208, andfurther in response to determining that no arc exist in node 232 forsymbol ‘C’, the processing unit stores, as a negative arc in cache, anindication of absence of arc with label ‘C’ in node 232 at location0x003 of cache 208. Processing element reprocesses symbol ‘C’ using rootnode 222. Root node 222 does not contain any arc for symbol ‘C’ and itis implicit arc pointing back to root node 222. And, hence symbol ‘C’ isconsumed.

At time t3, the processing unit may again evaluate symbol ‘A’ againstthe node 222 representation (e.g., stored in a cache, external memory,buffer memory, or other memory). In addition to evaluating symbol ‘A’against the representation of node 222, the processing unit may furtherevaluate the current symbol in view of the effective and negative arcsstored in cache 208. In response to determining that the symbol ‘A doesmatch a label of an effective stored at location x0001 in cache 208, theprocessing unit uses the effective arc from cache 208 to finishevaluating symbol ‘A’.

Similarly, at time t4, the processing unit may again evaluate symbol ‘B’against the node 222 representation (e.g., stored in a cache, externalmemory, buffer memory, or other memory). In addition to evaluatingsymbol ‘B’ against the representation of node 224, the processing unitmay further evaluate the current symbol in view of the effective andnegative arcs stored in cache 208. In response to determining that thesymbol ‘B’ does match a label of an effective stored at location x0002in cache 208, the processing unit uses the effective arc from cache 208to finish evaluating symbol ‘B’.

Finally, at time t5, the processing unit may evaluate symbol ‘D’ againstthe node 232 representation (e.g., stored in a cache, external memory,buffer memory, or other memory). In addition to evaluating symbol ‘D’against the representation of node 232, the processing unit may furtherevaluate the current symbol in view of the effective and negative arcsstored in cache 208. In response to determining that the symbol ‘D’ doesnot match a label of any effective or negative arcs stored in cache 208,and further in response to determining that the symbol matches a labelof an arc from node 232 to node 244, the processing unit stores, as aneffective arc, an indication of an arc with label ‘D’ from node 232 tonode 244 at location x004 of cache 208.

In this way, the processing unit may evaluate a payload moreefficiently. By caching effective or negative arcs, the processing unitcan potentially avoid performing repetitive, slower memory accesses, toevaluate current symbols against node representations that have alreadybeen used in previous evaluations, and may be stored outside of thecache. In other words, caching effective and negative arcs may enablethe processing unit to evaluate a current payload more quickly andefficiently by relying on previous evaluations of previous currentsymbols in a current payload.

In some examples, the processing unit may refrain from storing effectiveor negative arcs if a node representation for a current node is storedin an intermediate memory, e.g., buffer memory, rather than externalmemory. In other words, if a cache miss of a particular effective arcwill only result in an access to the intermediate memory, rather thanexternal memory, the processing unit may save space in cache 208 and notstore effective or negative arcs associated with that noderepresentation. Conversely, when a node representation is stored inexternal memory as opposed to a faster intermediate memory, theprocessing unit may perform effective and negative arc caching asdescribed above.

Turning to FIG. 10E, RegEx accelerator 200, in some examples, cacheseffective arcs without ever caching any negative arcs. Instead, RegExaccelerator 200 may cache resulting arcs as effective arcs, instead ofstoring negative arcs.

For example, similar to FIG. 10D, FIG. 10E includes views of cache 208at different points in time, relative to different cache keys derivedfrom a payload. For example, cache 208 at time t0 is empty or cold andincludes zero effective arcs. At subsequent times, t1-t5, cache 208includes information about the effective arcs that have been traversedduring the subsequent times. The payload being evaluated in FIG. 10E isincludes the string “ABCABC”.

As shown in FIG. 10E, at time t0, a processing unit of one of DFAengines 206 may evaluate symbol ‘A’ against the root node 222representation (e.g., stored in a cache, external memory, buffer memory,or other memory). In response to determining that the symbol matches anarc from node 222 to node 224, the processing unit stores, as aneffective arc, an indication of an arc with label ‘A’ from node 222 tonode 224 at location x001 of cache 208.

At time t1, the processing unit may evaluate symbol ‘B’ against the node224 representation (e.g., stored in a cache, external memory, buffermemory, or other memory). In addition to evaluating symbol ‘B’ againstthe representation of node 224, the processing unit may further evaluatethe current symbol in view of the effective arcs stored in cache 208. Insome examples, the processing unit may evaluate a current symbol againstcached effective arcs prior, concurrent, or subsequent to evaluating thesymbol against a current node representation. In other words, once cache208 is no longer cold, the processing unit may access cache 208 before,while, or after accessing the node representation. In response todetermining that the symbol ‘B’ does not match a label of any effectivearcs stored in cache 208, and further in response to determining thatthe symbol matches a label of an arc from node 224 to node 232, theprocessing unit stores, as an effective arc, an indication of an arcwith label ‘B’ from node 224 to node 232 at location x002 of cache 208.

At time t2, the processing unit may evaluate symbol ‘C’ against the node232 representation (e.g., stored in a cache, external memory, buffermemory, or other memory). In addition to evaluating symbol ‘C’ againstthe representation of node 232, the processing unit may further evaluatethe current symbol in view of the effective arcs stored in cache 208. Inresponse to determining that the symbol ‘C’ does not match a label ofany effective arcs stored in cache 208, and further in response todetermining that no arc exist in node 232 for symbol ‘C’, the processingelement reprocesses symbol ‘C’ using root node 222. Root node 222 doesnot contain any arc for symbol ‘C’ and therefore implicitly points backto root node 222. The processing unit stores as an effective arc (i.e.,a resulting arc), an indication of an arc with label ‘C’ from node 323to node 222 at location x003 of cache 208. And, hence symbol ‘C’ isconsumed.

At time t3, the processing unit may again evaluate symbol ‘A’ againstthe node 222 representation (e.g., stored in a cache, external memory,buffer memory, or other memory). In addition to evaluating symbol ‘A’against the representation of node 222, the processing unit may furtherevaluate the current symbol in view of the effective arcs stored incache 208. In response to determining that the symbol ‘A does match alabel of an effective stored at location x0001 in cache 208, theprocessing unit uses the effective arc from cache 208 to finishevaluating symbol ‘A’.

Similarly, at time t4, the processing unit may again evaluate symbol ‘B’against the node 222 representation (e.g., stored in a cache, externalmemory, buffer memory, or other memory). In addition to evaluatingsymbol ‘B’ against the representation of node 224, the processing unitmay further evaluate the current symbol in view of the effective arcsstored in cache 208. In response to determining that the symbol ‘B’ doesmatch a label of an effective stored at location x0002 in cache 208, theprocessing unit uses the effective arc from cache 208 to finishevaluating symbol ‘B’.

Finally, at time t5, the processing unit may again evaluate symbol ‘C’against the node 232 representation (e.g., stored in a cache, externalmemory, buffer memory, or other memory). In addition to evaluatingsymbol ‘C’ against the representation of node 232, the processing unitmay further evaluate the current symbol in view of the effective arcsstored in cache 208. In response to determining that the symbol ‘C’ doesmatch a label of an effective stored at location x0003 in cache 208, theprocessing unit uses the effective arc from cache 208 to finishevaluating symbol ‘C’.

Various examples have been described. These and other examples arewithin the scope of the following claims.

What is claimed is:
 1. A processing device comprising: a memoryconfigured to store at least a portion of a deterministic finiteautomata (DFA) graph; a cache configured to store at least one of: oneor more effective arcs or one or more negative arcs; and a DFA engineimplemented in circuitry comprising a processing unit configured toevaluate a payload by at least: determining whether a current symbol ofthe payload matches a label of any of the one or more effective arcs orthe one or more negative arcs associated with a current node of the DFAgraph that are stored in the cache; responsive to determining that thecurrent symbol does not match a label of any one of the one or moreeffective arcs or any one of the one or more negative arcs associatedwith the current node of the DFA graph, determining whether the currentsymbol matches a label of any arc associated with the current node ofthe DFA graph that is stored in the memory; and responsive todetermining that the current symbol matches a label of a particular arcassociated with the current node of the DFA graph that is stored in thememory, storing the particular arc in the cache as a new effective arcof the one or more effective arcs; and using the particular arc toevaluate the payload.
 2. The processing device of claim 1, wherein theprocessing unit is further configured to evaluate the payload by atleast: responsive to determining that the current symbol matches thelabel of a particular effective arc from any of the one or moreeffective arcs associated with the current node of the DFA graph thatare stored in the cache, use the particular effective arc to evaluatethe current symbol of the payload.
 3. The processing device of claim 1,wherein the processing unit is further configured to evaluate thepayload by at least: responsive to determining that the current symboldoes not match the label of any arc associated with the current node ofthe DFA graph that is stored in the memory, storing a new negative arcin the cache with the one or more negative arcs, wherein the newnegative arc indicates that the current symbol does not match the labelof any arc associated with the current node of the DFA graph; andevaluating the current symbol using a reference node associated with theDFA graph.
 4. The processing device of claim 1, wherein the processingunit is further configured to: responsive to determining that thecurrent symbol matches the label of a particular arc from any of the oneor more negative arcs, evaluate the current symbol using a referencenode associated with the DFA graph.
 5. The processing device of claim 1,wherein the processing unit is further configured to: responsive todetermining that a different current symbol of the payload matches thelabel of the new effective arc stored in the cache, using the neweffective arc to evaluate the different current symbol.
 6. Theprocessing device of claim 1, wherein the processing unit is furtherconfigured to evaluate the payload by at least: responsive todetermining that the current symbol does not match the label of any arcassociated with the current node of the DFA graph that is stored in thememory, determining whether the current symbol matches a label of anyarc associated with the reference node of the DFA graph; and responsiveto determining that the current symbol matches a label of a particulararc associated with the reference node of the DFA graph, storing theparticular arc in the cache as a new effective arc of the one or moreeffective arcs associated with the current node of the DFA graph; andusing the particular arc to evaluate the payload.
 7. The processingdevice of claim 6, wherein the processing unit is further configured toevaluate the payload by at least: responsive to determining that thecurrent symbol does not match the label of any arc associated with thereference node of the DFA graph, forcing an implicit match of currentsymbol with an imaginary arc pointing back to the reference node of theDFA graph; and storing the imaginary arc in the cache as a new effectivearc of the one or more effective arcs associated with the current nodeof the DFA graph; and using the imaginary arc to evaluate the payload.8. A method, performed by a processing unit of a deterministic finiteautomata (DFA) engine of a processing device, for evaluating payloads,the method comprising: determining whether a current symbol of a payloadmatches a label of any of one or more effective arcs or one or morenegative arcs associated with a current node of a DFA graph that arestored in a cache of the processing device; responsive to determiningthat the current symbol does not match a label of any one of the one ormore effective arcs or any one of the one or more negative arcsassociated with the current node of the DFA graph, determining whetherthe current symbol matches a label of any arc associated with thecurrent node of the DFA graph that is stored in a memory of theprocessing device; and responsive to determining that the current symbolmatches a label of a particular arc associated with the current node ofthe DFA graph that is stored in the memory, storing the particular arcin the cache as a new effective arc of the one or more effective arcs;and using the particular arc to evaluate the payload.
 9. The method ofclaim 8, wherein evaluating the payload further comprises: responsive todetermining that the current symbol matches the label of a particulareffective arc from any one of the one or more effective arcs associatedwith the current node of the DFA graph that are stored in the cache,using the particular effective arc to evaluate the current symbol of thepayload.
 10. The method of claim 8, wherein evaluating the payloadfurther comprises: responsive to determining that the current symboldoes not match the label of any arc associated with the current node ofthe DFA graph that is stored in the memory, storing a new negative arcin the cache with the one or more negative arcs, wherein the newnegative arc indicates that the current symbol does not match the labelof any arc associated with the current node of the DFA graph; andevaluating the current symbol using a reference node associated with theDFA graph.
 11. The method of claim 8, further comprising: responsive todetermining that the current symbol matches the label of a particulararc from any of the one or more negative arcs, evaluating the currentsymbol using a reference node associated with the DFA graph.
 12. Themethod of claim 8, further comprising: responsive to determining that adifferent current symbol of the payload matches the label of the neweffective arc stored in the cache, using the new effective arc toevaluate the different current symbol.
 13. A processing devicecomprising: a memory configured to store: at least a portion of adeterministic finite automata (DFA) graph, the DFA graph comprising aplurality of nodes, each of the nodes having zero or more arcs with eachof the zero or more arcs including a respective label and pointing to arespective subsequent node of the plurality of nodes, at least one ofthe plurality of nodes comprising a match node; one or more arcs of areference node representation for the at least the portion of the DFAgraph; and zero or more arcs of a respective delta node representationfor each node of the at least the portion of the DFA graph other thanthe reference node, the zero or more arcs of each respective delta noderepresentation defining arcs that are not defined by the reference noderepresentation; and a DFA engine implemented in circuitry comprising aprocessing unit configured to evaluate a payload by at least: accessingthe one or more arcs of reference node representation and the zero ormore arcs of the respective delta node representation that is associatedwith a current node to determine whether a symbol of the payload matchesat least one of: a label of any of the one or more arcs of the referencenode representation; and a label of any of the zero or more arcs of therespective delta node representation that is associated with the currentnode; and responsive to determining that the symbol of the payloadmatches a label of a particular arc of the one or more arcs of thereference node representation or the zero or more arcs of the respectivedelta node representation that is associated with the current node,using the particular arc to evaluate the payload.
 14. The processingdevice of claim 13, wherein the processing unit is further configured toevaluate the payload by at least: responsive to determining that thesymbol of the payload does not match the label of any of the one or morearcs of the reference node representation or the zero or more arcs ofthe respective delta node representation that is associated with thecurrent node, processing the symbol using an imaginary arc for thesymbol, the imaginary arc pointing back to the reference node.
 15. Theprocessing device of claim 13, wherein the processing unit is furtherconfigured to evaluate the payload by accessing the memory in responseto identifying the one or more arcs of the reference node representationthat are stored in a cache of the memory.
 16. The processing device ofclaim 15, wherein the processing unit is further configured to evaluatethe payload by at least: responsive to determining that the symbol ofthe payload matches a label of a particular arc of the one or more arcsof the reference node representation but does not match any of the zeroor more arcs of the respective delta node representation that isassociated with the current node, processing the symbol using theparticular arc of the reference node.
 17. The processing device of claim15, wherein the processing unit is further configured to evaluate thepayload by at least: responsive to determining that the symbol of thepayload matches a label of a particular arc of the zero or more arcs ofthe respective delta node representation, processing the symbol usingthe particular arc of the respective delta node.
 18. The processingdevice of claim 15, wherein the processing unit is further configured toevaluate the payload by refraining from accessing the memory in responseto determining that the one or more arcs of the reference noderepresentation are not stored in a cache of the memory.
 19. Theprocessing device of claim 18, wherein the processing unit is furtherconfigured to evaluate the payload by at least: responsive todetermining that the symbol of the payload matches a label of aparticular arc of the zero or more arcs of the respective delta noderepresentation that is associated with the current node, processing thesymbol using the particular arc of the respective delta node.
 20. Theprocessing device of claim 18, wherein the processing unit is furtherconfigured to evaluate the payload by at least: responsive todetermining that the symbol of the payload does not match any of thezero or more arcs of the respective delta node representation,processing the symbol using the reference node.
 21. The processingdevice of claim 13, wherein: the memory comprises multiple input portsand multiple output ports for enabling simultaneous access to the one ormore arcs of the reference node representation and the zero or more arcsof the respective delta node representation that is associated with thecurrent node; the memory is configured to: output, via a first outputport from the multiple output ports, a first result determined based ona first input to a first input port from the multiple input ports; andoutput, via a second output port from the multiple output ports, asecond result determined based on a second input to a second input portfrom the multiple input ports; and the processing unit is configured to:determine, based on the first result, whether the symbol of the payloadmatches the label of any of the one or more arcs of the reference noderepresentation; and determine, based on the second result, whether thesymbol of the payload matches a label of any of the zero or more arcs ofthe respective delta node representation that is associated with thecurrent node.
 22. The processing device of claim 13, wherein: the memorycomprises at least one first memory for enabling access to the one ormore arcs of the reference node representation and at least one secondmemory for enabling access to the zero or more arcs of the respectivedelta node representation that is associated with the current node,wherein the at least one first memory and the at least one second areconfigured to enable the processing unit to simultaneous access the oneor more arcs of the reference node representation stored in the at leastone first memory and the zero or more arcs of the respective delta noderepresentation that is associated with the current node stored in the atleast one second memory.
 23. The processing device of claim 13, wherein:the memory further comprises a cache configured to store at least oneof: one or more effective arcs or one or more negative arcs; and theprocessing unit is further configured to evaluate the payload by atleast: prior to speculatively accessing the one or more arcs ofreference node representation and the zero or more arcs of therespective delta node representation that is associated with the currentnode, determining whether the symbol of the payload matches a label ofany of the one or more effective arcs or the one or more negative arcsstored in the cache that are associated with the current node;responsive to determining that the symbol does not match a label of anyof the one or more effective arcs or the one or more negative arcsstored in the cache that are associated with the current node, storingthe particular arc in the cache as a new effective arc of the one ormore effective arcs in response to determining that the symbol of thepayload matches the label of the particular arc of the one or more arcsof the reference node representation or the zero or more arcs of therespective delta node representation that is associated with the currentnode.